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ABSTRACT 


It  is  essential  that  businesses  continually  improve  their  automated  information 
systems  (AIS)  to  support  the  changing  needs  of  the  organization.  The  Air  Force  civil 
engineering  organization  is  no  exception,  and  they  have  drastically  improved  their 
Automated  Civil  Engineer  System  (ACES)  since  its  implementation  in  2000.  However, 
there  are  many  problems  associated  with  the  non-appropriated  funds  (NAF)  project 
programming  business  rules  within  ACES.  These  problem  areas  were  not  addressed 
until  recently  when  an  integrated  process  team  (IPT)  met  and  proposed  numerous 
changes  to  how  NAF  programming  is  accomplished  in  ACES.  This  research  effort, 
through  a  web-based  survey,  focuses  on  the  perceived  benefits  of  these  proposed  changes 
from  a  base-level  programming  perspective.  It  also  investigated  current  programming 
procedures  that  might  affect  how  well  the  proposed  changes  are  implemented  along  with 
NAF  and  ACES  training  issues.  Descriptive  statistics  were  used  to  answer  the  research 
questions  using  survey  responses  from  a  sample  size  of  35  base-level  programmers. 

The  results  indicated  that  programmers  “agree”  or  “strongly  agree”  that  the 
majority  of  changes  proposed  by  the  IPT  will  be  beneficial  in  improving  NAF 
programming  in  ACES.  However,  several  potential  problems  areas  might  surface,  due  to 
current  programming  procedures  at  base-level,  when  these  changes  are  implemented  into 
ACES.  Automatic  email  notifications  on  project  status,  electronic  attachments  to  the 
project  file,  and  use  of  non  ACES  templates  are  all  areas  of  concern  brought  up  in  this 
research  effort. 
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EVALUATION  OF  CURRENT  AUTOMATED 


CIVIL  ENGINEERING  SYSTEM  NON- 
APPROPRIATED  FUNDS  PROJECT  PROGRAMMING 
PROCEDURES 


1.  INTRODUCTION 


The  advance  in  computer  technology  and  information  systems  has  altered  the 
processes  and  activities  of  all  businesses  and  organizations  (45:1).  Modern  businesses 
must  consider  their  information  technology  and  automated  information  systems  (AISs)  as 
crucial  factors  in  obtaining  a  sustainable  competitive  advantage  (45:1).  With  greater 
dependence  on  technology,  companies  are  constantly  pressured  to  update  and  improve 
their  AISs  to  better  match  their  business  processes  (45: 1).  Thus,  it  is  essential  that 
businesses  continually  update  and  improve  their  AISs  to  better  meet  the  changing  needs 
of  the  organization.  However,  many  problems  arise  as  organizations  try  to  modify  their 
existing  AISs  or  implement  new  ones.  Restructuring  of  organizational  requirements,  new 
training  demands,  and  system  inefficiency  due  to  user  resistance  are  several  problems 
areas  that  are  encountered  by  organizations  when  they  implement  a  new  AIS.  To  help 
alleviate  these  potential  headaches,  businesses  must  create  sound  AIS  development  and 
execution  plans  that  help  ensure  an  easy  transition.  The  technological  changes  and 
increasing  reliance  on  AISs  facing  businesses  are  also  causing  the  Air  Force  to  redefine 
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its  structure  and  interactions  as  an  organization  (6:i).  Thus,  the  Air  Force  is  facing 
implementation  problems  as  it  develops  new  AISs  to  better  match  its  business  processes. 

1.1  Automated  Information  Systems 

In  order  to  manage  AISs  more  efficiently,  and  reduce  the  amount  of  transitional 
problems  encountered,  it  is  essential  that  organizations  have  a  plan  on  how  to  implement 
their  existing  and  new  AISs.  Proper  and  efficient  implementation  and  maintenance  of 
AISs  are  crucial  for  organizations  to  manage  data  and  secure  a  competitive  advantage 
(45:1).  The  most  widely  accepted  methodology,  which  has  evolved  over  several  decades, 
for  analyzing  and  designing  information  systems,  is  the  systems  development  life  cycle 
(SDLC)  (41:7;  45:22).  The  SDLC  is  an  iterative  process  that  generally  adheres  to  the 
following  phases:  preliminary  investigation,  analysis,  logical  design,  physical  design, 
implementation,  and  maintenance  (45:23).  However,  whenever  an  organization 
implements  a  new  system,  it  is  inevitable  that  they  will  encounter  problems  (45:53).  The 
system’s  end  users  typically  help  make  these  problems  known  to  the  development  team 
during  the  maintenance  phase  of  the  SDLC  (45:53).  End  users  are  considered  to  be 
anyone  who  interacts  with  the  AIS  in  the  context  of  his  or  her  work  within  the 
organization  (41:5).  Regardless  of  how  well  the  SDLC  process  is  followed,  problems 
with  the  system  will  occur  in  the  maintenance  phase  since  many  issues  are  not  apparent 
until  the  system  is  in  full  use  (45:53). 

Business  rules  are  one  of  the  main  aspects  of  AISs  that  are  periodically  revised 
after  a  system  has  been  implemented  in  an  organization,  which  usually  occurs  during  the 
SDLC  maintenance  phase  (25:744).  Business  rules,  which  are  interaction  constraints 
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between  data  and  the  appropriate  fields,  help  prevent  user  error  and  create  more  accurate 
and  consistent  data  for  an  organization  (25:744).  Therefore,  is  it  essential  that  business 
rules  be  reviewed  frequently  to  ensure  that  they  still  mirror  the  way  an  organization 
operates  in  its  real  environment. 

The  Air  Force  has  many  AISs  that  facilitate  the  completion  of  its  various 
organizational  missions.  One  of  the  more  recently  implemented  systems  is  the 
Automated  Civil  Engineer  System  (ACES).  ACES  will  eventually  support  all  Civil 
Engineering  (CE)  requirements  and  be  utilized  by  the  whole  Air  Force  Civil  Engineering 
community,  to  include  officers,  enlisted  members,  civilians,  and  contractors. 

1.2  Automated  Civil  Engineer  System  (ACES) 

ACES  is  a  new  AIS  that  was  first  introduced  into  CE  operations  in  2000  (49:43). 
All  levels  of  CE,  to  include  installations,  Major  Commands  (MAJCOMs),  and 
Headquarters  Air  Force  (Air  Staff),  use  ACES  to  conduct  daily  and  strategic  management 
of  CE  functions.  ACES  is  CE’s  next  generation  AIS  that  will  provide  the  “best  tools  for 
us  to  plan,  program,  design,  construct,  operate,  maintain,  and  dispose  of  our  installations 
and  to  perform  our  agile  combat  support  tasks  in  both  peace  and  war”  (6:i).  The 
technological  advances  in  AIS  will  allow  CE  to  effectively  execute  all  these  tasks  with 
ACES. 

ACES  was  created  to  replace  and  centralize  former  CE  management  information 
systems  to  include  the  Work  Infonnation  Management  System  (WIMS)  and  the  Interim 
Work  Information  Management  System  (IWIMS);  it  was  also  intended  to  convert  Disk 
Operating  System  (DOS)  programs  into  a  relational  database  system  (27: 1).  The 
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window  based  ACES  system  links  all  base  CE  squadron  functions  to  the  Air  Force  wide 
network  via  an  internet  connection  through  a  central  database  located  at  Gunter  Annex, 
Maxwell  Air  Force  Base,  Alabama  (27:1).  The  Oracle-based  relational  database  meets 
the  new  standards  for  Air  Force  database  management  systems  (40:2).  Oracle  is  the 
database  software  that  is  used  to  retrieve  and  analyze  the  data  within  ACES. 

The  ACES  system  will  eventually  have  eight  separate  working  modules  to 
support  all  CE  operations.  Five  of  the  modules  have  already  been  implemented  and  have 
shown  that  there  are  areas  requiring  improvement.  These  modules  are  currently  in  the 
SDLC  maintenance  phase  where  end  users  are  recommending  changes  to  the  existing 
business  rules  to  improve  the  effectiveness  of  the  system.  One  of  the  modules,  the 
ACES  Project  Management  Module  (ACES-PM),  has  encountered  numerous  problems 
since  it  transitioned  from  the  out-dated  IWIMS.  The  other  three  modules  are  still  being 
developed  and  tested  prior  to  implementation. 

The  ACES-PM  module  is  used  for  the  programming,  design,  and  construction  of 
Air  Force  projects.  When  CE  transitioned  to  the  use  of  ACES-PM  in  2001,  the  business 
rules  associated  with  the  preceding  IWIMS  were  carried  over  to  the  new  system. 
However,  these  business  rules  were  designed  to  support  specific  kinds  of  projects  to 
include  Military  Construction  (MILCON)  and  Operations  &  Maintenance  (O&M).  The 
corresponding  business  rules  did  not  support  non-appropriated  funds  (NAF)  projects, 
which  are  essential  in  supporting  Morale,  Welfare,  Recreation,  and  Services  (MWRS) 
facilities. 
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1.3  Non-Appropriated  Funds  (NAF) 

Non-appropriated  funds  (NAF)  are  self-generating  funds  received  from  Air  Force 
personnel  through  their  patronage  of  Army  and  Air  Force  Exchange  Service  (AAFES) 
outlets  and  activities  along  with  the  profit  created  from  commissary  surcharges  and  other 
service  activities  (23:2).  Appropriated  funds  (APF),  on  the  other  hand,  are  funds 
appropriated  from  Congress  for  each  fiscal  year.  NAF  represent  the  main  funding  source 
used  to  support  facilities  that  house  MWRS  activities  (22: 1).  Air  Force  policy  mandates 
that  all  revenue-generating  MWRS  activities  fund  their  facility  projects  partly  with  NAF, 
unless  APF  are  authorized  to  provide  the  funding  (22:1;  26:1).  NAF  projects  provide 
both  economic  and  MWRS  returns  for  service  members,  making  it  essential  that  these 
facilities  be  maintained  and  upgraded  when  needed  (23:2).  Air  Force  installations 
realize  the  importance  of  building,  renovating,  and  maintaining  MWRS  related  facilities 
and  annually  submit  numerous  NAF  projects  to  their  MAJCOM  to  compete  for  funds  to 
improve  and  build  MWRS  facilities.  In  fiscal  year  2002  alone,  a  total  of  $123  million 
construction  dollars  were  reported  to  Congress  for  NAF  related  facilities  (3:6). 

Getting  NAF  construction  projects  funded  entails  many  separate  steps  before 
achieving  approval  from  the  final  authorization  level.  The  first  step  in  the  programming 
process  is  to  detennine  whether  the  project  can  actually  use  NAF  for  funding.  NAF 
programs  are  broken  into  five  separate  facility  and  function  categories  to  help  detennine 
whether  NAF,  APF,  or  a  combination  of  the  two  can  be  used  when  programming  for 
different  types  of  work  (23:6).  The  five  categories  are:  Category  A-Mission  Sustaining 
Activities,  Category  B-Basic  Community  Support  Programs,  Category  C-Revenue 
Generating  programs,  Lodging  Fund  Facilities,  and  other  activities  (23:6-13). 
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After  it  is  determined  that  a  project  is  funded  with  NAF,  the  approval  and  funding 
process  is  initiated.  Many  unique  documents  and  approval  steps  are  required  before  the 
project  can  be  approved  for  funding.  Part  of  the  process  requires  installations  to  submit 
an  Internal  Needs  Validation  Study  (INVS)  along  with  the  DD  Form  1391  which  is 
created  within  the  ACES-PM  system  (3:32;  23:38-39).  The  project  package  is  then  sent 
to  the  MAJCOM  who  selects  projects  for  funding,  and  forwards  it  to  the  Air  Force 
Services  Agency  (AFSVA)  Facilities  Panel.  The  panel  selects  projects  to  partake  in  a 
formal  Needs  Assessment  Study  (NAS)  (3;  36).  Based  on  the  NAS,  the  panel 
recommends  projects  for  35  percent  design  and  requests  a  design  instruction  (DI)  from 
CE  at  the  Air  Staff  level  (3:5;  23:22;  36).  After  projects  reach  the  35  percent  design 
point,  they  are  prioritized,  and  forwarded  to  Services  at  the  Air  Staff  level  for 
recommended  funding  (5;  36).  The  selected  projects  are  returned  to  CE  at  Air  Staff  with 
a  request  to  issue  the  100  percent  design  instruction  (5;  23:22).  Once  the  project  design 
is  complete  it  is  advertised  and  subsequently  awarded  for  construction  (5;  23:22). 

The  NAF  programming  process  is  very  detailed  and  confusing  since  many  of  the 
funding  sources  and  authorization  levels  depend  on  the  scope  of  the  project  and  the 
category  within  which  it  falls.  However,  all  NAF  projects  are  initiated  when  base  level 
programmers  enter  the  initial  program  requirement  and  supporting  documents  into 
ACES-PM.  The  use  of  ACES-PM  is  essential  during  this  approval  process  to  efficiently 
report  project  requirements  and  validate  funding  to  the  approval  authorities.  The 
efficiency  and  accuracy  of  programming  initial  NAF  project  requirements  becomes 
important  as  both  MAJCOM  and  higher  headquarters,  to  include  the  Air  Force  Services 
Agency  (AFSVA),  define  Air  Force  priorities  and  use  NAF  programming  documents  to 
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advocate  project  funding.  To  accomplish  this  effectively,  ACES-PM  business  rules  for 
NAF  projects  should  match  the  actual  funding  process  to  the  maximum  extent  possible. 

The  NAF  programming  process  is  extremely  different  than  that  required  by 
MILCON  and  O&M  projects.  Therefore,  the  current  ACES-PM  business  rules  are  not 
structured  efficiently  to  support  NAF  programming.  In  fact,  numerous  base  level  and 
MAJCOM  project  programmers  have  reported  problems  and  suggested  more  efficient 
methods  for  managing  NAF  projects.  It  is  essential  that  ACES-PM  business  rules  should 
be  revised  to  better  match  the  actual  NAF  programming  process.  However,  the 
implementation  of  ah  ACES  modules  is  not  yet  complete  and  the  maintenance  currently 
being  perfonned  on  non-NAF  ACES-PM  problems  is  intense.  Thus  the  organization 
responsible  for  the  success  of  ACES,  the  Air  Force  Civil  Engineer  Support  Agency 
(AFCESA),  has  not  been  able  to  focus  enough  resources  on  correcting  ACES-PM’s 
current  business  rules  to  better  support  the  NAF  programming  process.  In  response,  the 
ACES-PM  leadership  recently  initiated  an  IPT  meeting  where  numerous 
recommendations  were  made  concerning  how  the  current  business  rules  could  be 
changed  to  correspond  better  with  NAF  programming.  However,  as  with  most  AISs  in 
the  SDLC  maintenance  phase,  there  is  always  room  for  improvement  as  end  users  figure 
out  better  ways  to  utilize  the  system.  Furthermore,  base-level  programmers  were  not 
represented  at  the  IPT  meeting. 

1.4  Research  Questions 

The  purpose  of  this  research  is  to  determine  whether  the  IPT’s  proposed  changes 
to  the  current  NAF  ACES-PM  programming  procedures  are  efficient  and  complete.  This 
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research  will  be  used  to  give  the  ACES-PM  IPT  recommendations  for  both  additions  and 
deletions  to  the  proposed  changes,  along  with  an  idea  of  how  well  these  changes  will  be 
accepted  within  the  Air  Force  base-level  programming  community.  The  research  will 
also  determine  whether  any  existing  base-level  programming  procedures  will  hinder  the 
implementation  of  the  IPT’s  proposed  business  rule  changes.  Finally,  the  research  looks 
at  the  current  training  situation  for  programmers  regarding  NAF  programming  in  general 
and  NAF  programming  in  ACES-PM  specifically.  In  doing  so,  this  research  effort 
attempted  to  answer  the  following  questions: 

1 .  How  well  accepted  is  each  of  the  proposed  ACES-PM  NAF  project  programming 
business  rule  changes  in  terms  of  the  Air  Force  base-level  programming 
community? 

2.  How  well  does  the  current  base-level  ACES-PM  programming  process  support 
the  new  ACES-PM  NAF  business  rule  changes? 

3.  How  well  trained  are  current  base-level  programmers  in  NAF  programming  and 
NAF  programming  in  ACES-PM? 

1.5  Research  Methodology 

A  web-based  survey  was  used  to  collect  the  data  necessary  for  this  research  effort. 
In  order  to  answer  the  research  questions  above,  quantitative  methods  to  include 
descriptive  statistics  were  used  to  evaluate  the  survey  responses.  The  survey  was 
administered  to  base-level  programmers  who  have  had  recent  experience  with 
programming  NAF  projects  in  ACES-PM.  The  survey  consisted  of  33  questions  that 
measured  several  constructs:  demographics  (grade/rank  and  MAJCOM),  the  level  of 
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training  in  regards  to  NAF  programming  and  programming  NAF  projects  in  ACES-PM, 
programming  procedures  that  could  affect  implementation  of  the  proposed  business  rule 
changes,  and  the  specific  business  rule  changes  proposed  by  the  IPT. 

The  survey  consists  of  the  following  types  of  questions:  multiple  choice,  fill-in- 
the -blank,  and  Likert  Scale.  All  questions  pertaining  to  the  proposed  business  rule 
changes  and  level  of  training  were  answered  using  the  Likert  Scale.  The  questions 
related  to  current  programming  procedures  used  both  the  Likert  Scale  and  multiple  choice 
questions.  The  demographics  were  determined  with  fill-in-thc-blank  questions. 

1.6  Scope  and  Limitations  of  Research 

This  research  concentrated  solely  on  the  ACES-PM  module,  which  is  used 
significantly  to  report  NAF  project  requirements.  The  scope  of  this  research  included 
only  the  business  rules  associated  with  NAF  project  programming  procedures  within 
ACES-PM  and  the  NAF  programming  process  in  general.  All  other  problems  with 
ACES,  ACES-PM,  or  the  NAF  programming  process  brought  up  by  the  survey 
participants  were  not  addressed.  Additionally,  the  results  of  this  research  effort  will  be 
aimed  towards  a  business  management  solution  instead  of  an  information  technology 
solution.  It  involves  finding  out  what  changes  to  the  current  programming  procedures 
and  business  rules  should  be  included  in  programming  NAF  projects  within  ACES-PM, 
not  the  code  and  logic  required  to  actually  change  the  AIS. 

This  research  only  covered  areas  of  ACES-PM  that  pertained  to  the  research 
questions  and  subsequent  survey  questions.  Base-level  programmers  with  NAF 
programming  experience  were  surveyed  to  detennine  appropriate  ways  to  improve  NAF 
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programming  in  ACES-PM,  specifically  in  regards  to  the  proposed  changes  made  by  the 
ACES-PM  IPT.  This  sample  selection  was  considered  adequate  for  the  research  effort 
since  most  initial  project  programming  occurs  at  base  level  as  facility  requirements  arise. 
The  sample  will  give  each  MAJCOM  and  higher  headquarters  agencies  a  better 
understanding  of  where  base-level  programmers  stand  on  how  to  improve  ACES-PM 
NAF  programming. 

1.7  Review  of  Chapters 

Chapter  2  provides  a  summary  of  the  appropriate  literature  pertaining  to 
information  systems,  ACES,  and  NAF  programming.  It  examines  the  history  of  ACES, 
its  responsible  organizations,  end  user  changes,  and  current  status.  Chapter  2  also 
discusses  the  process  and  rules  associated  with  NAF  programming  in  ACES-PM. 

Chapter  3  examines  the  methodology  used  for  answering  the  research  questions  along 
with  explaining  the  data  gathering  and  analysis.  Chapter  4  provides  the  results  of  the 
research  questions  and  details  the  responses  to  each  of  the  survey  questions.  It  also 
includes  the  statistical  analysis  results  of  the  survey.  In  conclusion,  Chapter  5 
summarizes  the  results  of  the  research,  discusses  limitations,  covers  future  research  areas, 
and  makes  recommendations  on  how  to  further  improve  NAF  programming  in  ACES- 
PM. 
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2.  BACKGROUND 


This  chapter  summarizes  the  literature  significant  to  this  research  effort.  The 
information  is  organized  into  five  main  sections:  1)  description  of  automated  information 
systems  (AIS)  to  include  its  development  and  role  in  the  Air  Force;  2)  information  about 
the  Automated  Civil  Engineer  System  (ACES)  and  its  current  status;  3)  the  general 
background  of  non- appropriated  funds  (NAF)  project  programming  and  its  funding 
process;  and  4)  the  current  NAF  programming  business  rules  within  the  Automated  Civil 
Engineer  System  Project  Management  Module  (ACES-PM).  This  literature  review  will 
allow  for  a  better  understanding  of  the  AIS  under  research,  the  requirements  for 
programming  NAF  projects,  and  how  NAF  projects  are  currently  managed  within  ACES- 
PM.  An  understanding  of  these  concepts  helps  create  a  general  knowledge  base  for 
ACES  and  NAF  information  which  allows  for  a  better  analysis  of  the  survey  responses 
and  results. 

2.1  Development  and  Importance  of  Automated  Information  Systems  (AISs) 

2.1.1  Background 

The  accelerating  technological  changes  in  automation  software,  communications, 
and  information  systems  are  changing  how  businesses  structure  their  organizations, 
conduct  business,  and  interact  (6:1).  The  more  efficiently  companies  can  store,  control, 
and  retrieve  data,  the  greater  the  advantage  they  will  have  over  their  competitors.  To 
control  information  more  efficiently,  most  management  informational  processes  are 
becoming  digital,  thereby  forcing  organizations  to  become  more  dependent  on  their 
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information  systems  and  the  people  who  develop  and  maintain  them  (45:xvii).  Therefore, 
the  management  of  infonnation  systems  should  be  approached  with  a  clear  and  efficient 
process  to  minimize  expenses  and  ensure  that  effective  AISs  are  implemented  within 
organizations. 

2,1.2  The  System  Design  Life  Cycle  (SDLC) 

In  order  to  acquire  a  consistent  methodology  for  creating  and  maintaining  the  AIS 
required  in  today’s  business  environment,  organizations  rely  heavily  on  the  System 
Design  Life  Cycle  (SDLC).  The  SDLC,  which  is  the  most  widely  accepted  approach  to 
the  analysis  and  design  of  AISs,  solves  an  existing  business  problem  by  redesigning 
information  systems  and/or  implementing  new  ones  (45:23).  The  phases  within  the 
SDLC  methodology  vary,  depending  on  the  source;  however,  most  include  some 
combination  of  the  phases  described  in  Table  1.  Each  phase  is  an  equally  important 
activity  that  helps  structure  and  guides  the  AIS  development  process  (45:23).  Although 
these  phases  are  described  in  discrete  phases,  they  are  typically  not  accomplished  as 
separate  steps  (41:7).  The  SDLC  phases  should  be  accomplished  as  simultaneously  as 
possible  to  speed  up  the  development  process  (41:7). 


12 


Table  1.  The  System  Development  Life  Cycle  (SDLC)  (41:7-12;  45:49-56) 


LIFE  CYCLE 
PHASE 

KEY  ACTIVITIES 

PRIMARY 

DELIVERABLES 

Preliminary 

Investigation 

Define  problem,  estimate  scope 
and  feasibility,  detennine  “go/no 
go”  decision 

General  problem  statement  and 
feasibility  report 

Analysis 

Analyze  the  problem,  business 
environment,  and  existing 
systems  to  determine 
requirements 

Formal  problem  statement  and 
requirements  definition  that 
meets  all  user  requirements 

Logical  Design 

Structure  all  requirements  to 
reflect  proposed  solution  to  the 
problem  and  validate 

Detailed  performance 
specifications  for  entire 
information  system 

Physical  Design 

Determine  hardware  and 
software  specifications,  training 
and  implementation  guidelines, 
and  preliminary  system  testing 

Final  feasibility  report, 
program  and  database 
structures,  and  implementation 
schedule 

Implementation 

Install  new  system,  verify  it  with 
end  users,  train  end  users,  and 
convert  all  data  to  new  system 

Fully  installed  system, 
converted  data  files,  and 
performance  test  metrics 

Maintenance 

Monitor  performance  and 
perform  requested  necessary 
changes  to  system 

Fully  functioning  system  with 
periodic  auditing  of  system  life 
cycle  costs 

It  is  always  better  to  find  mistakes  with  new  AISs  as  early  as  possible  in  the 
SDLC  process.  The  further  along  an  organization  is  in  the  SDLC  process,  the  more 
expensive  it  becomes  to  overcome  a  mistake  (45:56).  Some  researchers  believe  that  60 
percent  or  more  of  the  total  time  spent  on  infonnation  systems  is  on  the  maintenance  of 
the  existing  systems  (41:1 1).  This  percentage,  and  the  associated  cost,  increases  even 
more  when  planning  is  inadequate  during  the  early  stages  of  the  SDLC  (45:25).  Proper 
planning  during  the  preliminary  investigation,  analysis,  logical  design,  and  physical 
design  phases  will  help  alleviate  potential  problems  during  implementation  efforts.  If 
problems  with  the  system  arise  during  any  of  the  SDLC  phases,  the  development  team 
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must  back  track  to  find  the  problem  and  correct  it  (41:7).  However,  the  relative  cost  to 
fix  an  error  rises  exponentially  the  further  into  the  SDLC  process  that  it  takes  to 
recognize  the  problem  (45:25).  In  addition  to  high  costs,  poor  planning  can  also  lead  to  a 
large  maintenance  problem  that  will  take  more  resources  and  time  to  fix.  It  is  also 
important  to  ensure  proper  planning  occurs  so  that  the  end  users  do  not  lose  confidence  in 
a  new  system  and  reject  it. 

However,  no  matter  how  much  planning  is  put  into  a  new  management 
information  system,  problems  always  arise.  It  is  impossible  to  foresee  all  the  issues  that 
arise  when  a  new  or  updated  AIS  is  implemented.  One  potential  problem  deals  with  bugs 
or  “known  anomalies”  that  creep  into  the  system  only  after  the  system  is  implemented 
(41:1 1).  For  example,  when  anomalies  are  detected  in  software  applications  patches  or 
new  versions  are  released  to  correct  the  problem.  Other  problems  might  involve 
organizational  needs,  software  or  hardware,  or  the  level  of  technological  change  within  a 
company  (41:11).  A  common  example  includes  end  users  requiring  additional  features  or 
changes  to  current  business  rules  once  they  become  familiar  with  new  AISs  (41:1 1).  The 
AIS  business  rules  need  to  match  the  actual  organizational  process  to  the  maximum 
extent  possible  and  the  end  users  are  in  the  best  position  to  discover  inconsistencies 
(45:24).  Problems  such  as  these  are  usually  addressed  in  the  SDLC  maintenance  phase, 
unless  they  are  caught  by  the  development  team  in  an  earlier  phase.  In  the  maintenance 
phase,  once  end  users  become  familiar  with  a  new  AIS,  they  find  more  efficient  ways  to 
accomplish  tasks  that  are  not  currently  supported  by  the  system  (45:53).  Overall,  the 
amount  of  time  devoted  to  making  these  necessary  changes  during  the  maintenance  phase 
usually  depends  on  the  quality  of  work  executed  during  the  previous  five  phases  (45:54). 
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2.1.3  AIS  Importance  to  Civil  Engineering 

Air  Force  Civil  Engineering  is  responsible  for  all  the  real  property  owned  by  the 
Air  Force  to  include  facilities,  utilities,  and  land.  The  Air  Force  Civil  Engineering’s 
mission  is  “to  provide  the  bases,  infrastructure  and  facilities  necessary  to  support  the 
global  engagement  of  air  and  space  forces  across  the  spectrum  of  conflict”  (18).  In  order 
to  support  this  mission,  civil  engineering  (CE)  performs  a  wide  range  of  services  that  are 
organized  under  any  of  the  squadron’s  eight  flights  shown  in  Table  2. 

As  Table  2  indicates,  CE  is  responsible  for  a  large  number  of  functional  areas, 
each  of  which  demands  many  resources  to  accomplish  its  mission.  With  limited 
resources  that  compete  against  each  other,  CE  must  able  to  efficiently  store  and  retrieve 
large  amounts  of  data.  Since  this  data  supports  all  operations  within  CE  and  can  be 
shared  with  other  entities  that  affect  or  support  CE  operations,  it  is  important  that  CE 
develop  and  maintain  efficient  and  effective  AISs  (14:2).  The  requirement  for 
standardized  and  shared  information  is  applicable  not  only  at  a  specific  location  but 
throughout  the  entire  Air  Force  (6:i).  Consequently,  the  CE  leadership  and  the  Air  Force 
have  realized  the  importance  of  updating  its  AISs.  Specific  to  the  CE  community,  ACES 
was  developed  with  the  goal  being  to  meet  all  current  and  future  CE  information  system 
requirements  (6:i). 
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Table  2.  CE  Flight  Structure  (48:25) 


FUNCTIONAL  AREA 

DESCRIPTION  OF 
RESPONSIBILITIES 

Command  Section 

Unit  commander  and  unit  administration 
duties 

Engineering 

Plans,  designs,  and  executes  facility 
construction  and  repair 

Fire  Protection 

Protects  from  fire  as  well  as  manages 
hazardous  material  accident  response 

Operations  and  Maintenance 

Maintains  all  facilities  and  infrastructure  on 
the  installation 

Environmental  Services 

Ensures  compliance  with  environmental 
policy  as  well  as  reducing  pollution  and 
cleaning  contaminated  sites 

Military  Housing 

Manages  installation  housing  assets 

Explosive  Ordinance  Disposal 

Provides  explosive  relief  services  to  the 
installation  and  local  area 

Readiness  and  Disaster  Preparedness 

Trains  personnel  of  military  skills  as  well 
as  disaster  preparedness  training  and 
response 

Resources 

Manages  base  property  assets  as  well  as  CE 
finances,  manpower  and  equipment 

2.2  Automated  Civil  Engineer  System  (ACES) 

2.2.1  Background 

The  first  AIS  to  be  used  throughout  Air  Force  CE,  the  Base  Engineer  Automated 
System  (BEAMS),  was  implemented  in  the  early  1970s  (1:9).  BEAMS  was  not 
centralized,  used  only  “dumb  terminals,  and  required  punch  card  data  entry  (1:9). 

BEAMS  was  eventually  replaced  by  the  Work  Information  Management  System  (WIMS) 
in  the  mid  1980s  (1:9).  WIMS  operated  using  “dumb”  tenninals  and  personal  computers, 
a  closed  architecture,  and  flat-file  (non-relational)  databases  (1:9).  Both  BEAMS  and 
WIMS  operated  with  a  closed  non-relational  system  which  resulted  in  slow  data 
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responses  within  the  CE  field  (1:9).  The  transition  to  open  systems  was  initiated  when 
WIMS  migrated  to  the  Interim- WIMS  (IWIMS)  in  accordance  with  Defense 
Management  Review  Decision  (DMRD)  #  924,  which  mandated  that  all  legacy  systems 
migrate  to  open  systems  and  consolidate  resources  at  Defense  Enterprise  Computing 
Centers  (DECCs)  (1:6).  Legacy  systems,  like  BEAMS  and  WIMS,  are  information 
systems  that  have  been  used  for  a  long  time  with  a  mainframe  computer  (1:54).  As  a 
result  of  DMRD  #  924,  WIMS  was  moved  to  the  DECC  at  Gunter  Annex  and  IWIMS 
was  created  to  centralize  the  AIS  development  process.  However,  IWIMS  still  did  not 
meet  the  open  systems  requirement  outlined  in  DMRD  #  924.  Thus,  the  conversion  to  a 
true  relational  system,  and  the  elimination  of  the  outdated  IWIMS,  started  with  the 
development  of  ACES  ( 1 :9).  ACES  is  now  in  the  early  stages  of  meeting  the 
requirements  set  forth  in  DMRD  #  924  as  well  as  the  goals  and  objectives  articulated  by 
Air  Force  leadership  ( 1 :9). 

2.2.2  Purpose 

The  ACES  program  has  a  distinct  vision  and  mission,  along  with  specific  goals 

and  objectives  to  ensure  all  higher  headquarters’  requirements  are  met.  The  CE  AIS 

Modernization  Program’s  vision  for  ACES  is  to: 

Improve  business  processes  through  a  transition  of  the  IWIMS  framework 
into  a  relational  database  linked  to  graphical  application,  supporting  the 
full  range  of  operational  and  contingency  responsibilities.  The  envisioned 
system  will  be  appropriately  integrated  and  standardized  to  share  data  with 
other  entities  that  affect  or  support  Civil  Engineer  Operations  (1:13). 
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ACES’s  mission  is  to  “establish  a  user-friendly,  flexible,  and  high  perfonnance  On-Line 
Transactional  Processing  and  On-Line  Analytical  Processing  environment  for  Air  Force 
warfighters”  (1:13).  The  main  goals  and  objectives  of  the  ACES  program  are  to  (1:13): 

1 .  Provide  sustainment  of  existing  fields, 

2.  Spiral  development  of  ACES  (conversion  of  non-relational  IWIMS  to 
relational  database), 

3.  Insert  new  technology  (web  and  AF  portal),  and 

4.  Comply  with  Air  Force  and  Department  of  Defense  (DoD)  mandates. 

The  strategy  of  the  ACES  program  is  to  eventually  replace  IWIMS  to  support  all  CE 
functions  with  an  incremental  approach  while  integrating  the  AF’s  evolving  infonnation 
technology  architecture  (1:14). 

The  ACES  system  provides  direct  CE  information  management  support  to  all  Air 
Force  units  during  all  types  of  operations  (27:3).  Its  data  entry  and  retrieval  processes  are 
accomplished  through  a  web-based  environment  that  utilizes  existing  infrastructure, 
operating  systems,  and  computer  systems  (27:5).  Oracle,  NT  web  servers,  and  a  UNIX 
platform  support  the  ACES  database  on  the  web  (27:4). 

The  ACES  system  integrates  base-level  CE  functions  and  allows  higher 
headquarters  to  view  the  information  in  real-time  (27:3).  It  is  designed  to  support  base- 
level  and  higher  headquarters  CE  functions  in  day-to-day  operations  ( 1 :6).  It  also 
functions  as  an  interoperable  automated  system  that  expedites  civil  engineering  support 
during  peacetime  and  contingency  operations  (1:6).  ACES  allows  Civil  Engineers  to 
spend  more  time  analyzing  information  in  a  user  friendly,  graphical  format  and  less  time 
inputting  and  searching  for  information  (6:1).  ACES  will  eventually  be  compatible  with 
other  Air  Force  personnel  management  AISs  (27:6). 
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2.2.3  Implementation 


The  ACES  system  is  being  implemented  in  eight  separate  phases  called  modules, 
as  shown  in  Table  3.  Each  module  aligns  with  and  supports  one  of  the  eight  flights  (i.e., 
primary  functions)  in  a  typical  CE  organization  (1:7).  Along  with  supporting  the 
functional  areas,  ACES  must  also  provide  accountability  for  audit  requirements  and  allow 
data  to  be  obtained  from  other  systems  (1:7). 

Table  3.  ACES  Modules  Description  (27:7-8;  43) 


FLIGHT 

FUNCTION 

Engineering  Flight 

Project  Management  and  Programming, 
Design,  Construction,  and  Restoration 
and  Cleanup  (ACES-PM) 

Housing  Flight 

Military  Family  Housing  (ACES-HM) 

Resources  Flight 

Real  Property  (ACES-RP) 

Readiness  Flight 

Personnel  Training  and  Readiness 
Equipment  Management  (ACES-PR) 

Fire  Flight 

Incident  Response  Management  (ACES- 
FD) 

Operations  Flight 

Work  Control  and  Financial 

Management,  and  the  CE  Material 
Acquisition  System  (CEMAS)  (ACES- 
OPS) 

Environmental  Flight 

Environmental  Management  (ACES-EM) 

Explosive  Ordinance  Flight 

Incident  Response  Management  (ACES- 
EO) 

2.2.4  Responsible  Organizations 

The  primary  organizations  involved  with  ACES  are  the  Air  Force  Civil  Engineer 
Support  Agency  (AFCESA),  located  at  Tyndall  Air  Force  Base,  Florida,  and  the  Standard 
System  Group  (SSG),  located  at  Gunter  Annex,  Maxwell  Air  Force  Base,  Alabama. 
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AFCESA  is  responsible  for  providing  sufficient  resources  and  professional  support  to 
help  Air  Force  Civil  Engineering  succeed  in  base  and  contingency  operations  (12).  One 
of  its  responsibilities  includes  the  task  of  providing  “the  guidance  for  initiating, 
approving,  and  authorizing  the  implementation  of  operationally  approved 
requirements/changes  to  hardware,  software,  firmware,  and  communication  configuration 
items  of  ACES”  (28).  AFCESA  also  facilitates  the  organization  of  the  Automated 
Steering  Group  (ASG),  Configuration  Control  Board  (CCB),  and  the  Integrated  Process 
Teams  (IPTs)  (14:2).  To  complement  AFCESA,  the  mission  of  the  SSG  is  to  “provide 
and  support  secure  combat  support  information  systems  and  networks  for  the  Air  Force 
and  DoD  components”  (17).  The  primary  functions  of  these  organizational  units  and  the 
working  relationship  between  them  are  shown  in  Figure  1  and  will  be  explained  in  the 
following  paragraphs. 
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Figure  1.  ACES  Organizational  Structure  (6:5) 


2.2.4.1  Air  Force  Civil  Engineer  Support  Agency 

The  Technology  Integration  Division  (TID)  within  AFCESA’s  Operations 
Support  Directorate  serves  as  the  point  of  contact  for  all  ACES  development  issues  and  is 
the  ACES  Program  Management  Office  (1:6;  12).  The  TID  is  responsible  for  managing 
the  sustainment  and  developmental  funding  of  ACES,  the  strategic  plan,  and  the 
interactions  with  stakeholders  to  ensure  the  AIS  meets  DoD  and  Air  Force  infonnation 
technology  directives  (6:6).  It  also  defines  the  procedures  to  implement  a  change  to  the 
AIS  once  a  specific  ACES  module  has  been  implemented. 

2.2.4.2  Standard  Systems  Group 

When  the  Air  Force  originally  sought  to  replace  WIMS  with  ACES,  they  awarded 
the  contract  to  design  and  create  the  ACES  program  to  Martin-Marietta  (43).  However, 
the  contract  failed  to  meet  Air  Force  requirements,  and  the  Standard  Systems  Group 
(SSG)  is  now  completing  the  design  of  ACES  (16).  They  are  the  System  Program  Office 
(SPO)  for  the  completion  of  ACES  (1:6).  The  SSG  is  responsible  for  writing  the  ACES 
program  with  AFCESA’s  direction  along  with  being  the  technical  advisor  to  the  CCB  and 
ASG  (6:5).  The  SSG  “designs,  builds  or  buys,  installs,  integrates  and  supports 
information  systems  necessary  to  provide  the  warfighter  the  right  combat  support 
information  in  the  right  place  at  the  right  time”  (17).  Their  vision  is  to  eventually 
become  DoD’s  center  of  choice  for  developing  and  maintaining  information  systems 
(17).  The  SSG  also  developed  a  version  of  the  SDLC,  called  the  Systems  Engineering 
Process  (SEP),  which  will  be  described  further  in  Section  2.2.7. 
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2.2.43  Automated  Steering  Group 

The  Automated  Steering  Group  (ASG),  which  is  chaired  by  the  Deputy  Air  Force 
Civil  Engineer,  acts  as  the  liaison  between  senior  leadership,  system  developers,  and  end 
users  (6:5;  45:15).  This  group  provides  strategic  direction  and  resolves  conflict  between 
the  separate  organizations  (6:6).  The  ASG  is  also  responsible  for  controlling  ACES’s 
budget  (14:4).  Members  of  the  ASG  include  Air  Force  Institute  of  Technology  (AFIT), 
MAJCOM,  and  Air  Force  C4  Information  Command  (AFCIC)  representatives  (14:4). 

2.2.4.4  Configuration  Control  Board 

Besides  involvement  in  the  AIS  change  approval  process,  the  Configuration 
Control  Board  (CCB)  provides  tactical  direction  for  ACES  development  and  coordinates 
IPT  actions  (6:5).  They  are  the  technical  arm  of  the  ASG  (14:4).  The  board  approves  all 
user  initiated  changes  and  prioritizes  them  for  SSG  implementation.  The  CCB  also 
defines  functional  requirements  and  coordinates  implementation  and  training  issues  with 
the  other  organizations  (6:7).  The  CCB  is  chaired  by  AFCESA  and  has  technical 
members  representing  AFIT,  MAJCOMs  and  the  SSG  (28:13). 

2.2.4.5  Integrated  Process  Teams 

Each  ACES  module  has  a  designated  integrated  process  team  (IPT)  that  defines 
functional  business  requirements  and  develops  solutions  to  problems  typically 
encountered  during  the  SDLC  maintenance  phase.  Their  role  is  to  serve  as  the  functional 
working  group  representing  the  CE  community’s  ACES  requirements  (14:5).  The  IPTs 
are  empowered  to  “seek  alternative  automated  solutions  which  may  include  redefining 
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existing  business  processes”  or  rules  (14:5).  They  also  recommend  valid  user  initiated 
changes  to  the  CCB  (28:7). 

2.2.4.6  Other  Organizations 

Base  level  Civil  Engineering  helps  during  the  SDLC  maintenance  phase  by 
participating  in  business  rule  improvement  workshops  and  forwarding  end  user  problems 
up  to  the  IPTs  for  further  examination.  They  help  identify  specific  AIS  initiatives  to  their 
respective  MAJCOM  (6:6).  CE  squadrons  are  also  responsible  for  the  purchase  and 
maintenance  of  its  ACES  hardware  and  automation  software  (6:6). 

Training  and  education  fall  under  the  responsibility  of  AFIT  and  Sheppard 
Tactical  Training  Center  (TTC)  (1:32;  14:4).  AFIT  along  with  the  SSG  is  developing 
just-in-time  training  that  will  be  implemented  using  a  web-based  training  program  ( 1 :32). 
AFIT  is  also  involved  with  the  allocation  of  resources  and  identifying  training 
requirements  (6:5). 

2.2.5  User  Initiated  Changes 

The  ASG,  CCB,  and  IPTs  are  the  main  organizations  that  deal  with  user  requests 
to  add  or  change  requirements  within  ACES.  This  change  process,  which  is  initiated  by 
an  AF  Form  3215,  the  C4  Systems  Requirements  Document  (CSRD),  is  shown  in  Figure 
2  (28:7).  C4  stands  for  Command,  Control,  Communications,  and  Computer.  The 
change  process  begins  when  ACES  users  submit  the  CSRD  to  their  MAJCOM  CCB 
representative  requesting  a  new  requirement  or  change  to  the  system  (28:4).  The  CSRD 
is  the  reporting  tool  ACES  end  users  utilize  to  voice  their  concerns  and  suggest  further 
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Figure  2.  Civil  Engineer  Automated  Request  Approval  Process  (28:7) 
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improvements  to  the  AIS.  Any  change  that  might  impact  the  integrity  or  compatibility  of 
ACES  must  be  initiated  with  an  AF  Form  3215  (28:3).  Any  end  user  can  submit  an  AF 
Form  3215,  but  that  does  not  necessarily  mean  that  the  MAJCOM  CCB  will  approve  the 
request  and  forward  it  to  AFCESA  (28:6).  To  fill  out  the  fonn,  the  end  user  or  requesting 
agency  has  to  describe  and  justify  the  new  AIS  requirement  (28:4).  Since  the  release  of 
ACES-PM’s  new  version  in  September  2003,  the  IPT,  CCB,  ASG,  and  SSG  have 
incorporated  numerous  end  user  requirements  using  this  change  process  (44).  During 
that  time  period,  the  groups  reviewed  over  200  CSRDs  (44).  This  AIS  improvement,  via 
user  input,  is  completed  in  the  maintenance  phase  of  the  SDLC,  or  the  customer  support 
step  under  the  SEP. 

2.2.6  Regulations,  Guidance,  and  Standards 

Air  Force  Instruction  (AFI)  32-1019,  Automated  Civil  Engineer  Information 
Management,  which  is  still  in  draft  fonn,  will  be  the  main  CE  guidance  dealing  with  the 
management  of  civil  engineer  automation  procedures  and  regulations.  It  sets  the 
standards  for  appropriately  integrating  ACES  into  CE  operations  and  sharing  data  with 
other  information  systems  (14:2).  The  draft  AFI  states  that  standard  business  rules  must 
be  incorporated  into  the  AIS  to  provide  flexibility  in  data  collection  and  analysis  (14:2). 
More  specifically,  AFI  32-1019  “establishes  an  organization  and  related  responsibilities 
for  providing  overarching  oversight,  technical  review,  and  functional  area  expertise  for 
defining,  fielding,  and  training  the  Civil  Engineer  data  automation  tools”  (14:1).  The  AFI 
creates  the  ASG,  CCB,  and  IPT  for  each  module  and  defines  their  responsibilities  and 
tasks  (14.1).  The  ACES  system  will  enable  the  standardized  sharing  of  data  with  all 
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entities  that  affect  or  support  CE  operations,  while  reducing  cost  for  maintenance  and 
duplicated  efforts  (14:2). 

AFI  32-1019  also  requires  compliance  with  DoD  Manual  8320. 1-M,  Data 
Administration,  and  DoD  Manual  8320. 1  -M- 1 ,  Data  Standardization  Procedures,  when 
interfacing  data  into  and  from  ACES  (14:6).  DoD  Manual  8320. 1-M  outlines  the  goals  of 
the  DoD  Data  Administration  Program  as  listed  below  (9:3-6): 

1 .  Have  a  centrally  controlled  DoD  wide  data  repository, 

2.  Standardize  data, 

3.  Use  of  common  procedures  and  automation  tools, 

4.  Use  quality  data  in  all  decision  making, 

5.  Have  education,  training,  and  consultation  services,  and 

6.  Have  effective  infrastructure. 

In  addition  to  these  broad-based  goals,  DoD  Manual  8320.1-M-l  describes  how  AISs 
must  standardize  its  data  to  accomplish  the  following  objectives  (10:2-3): 

1 .  Improve  data  sharing  across  DoD  functional  areas, 

2.  Control  Data  redundancy, 

3.  Minimize  data  processing  and  storage  in  infonnation  systems, 

4.  Improve  data  accuracy  and  consistency  throughout  the  DoD,  and 

5.  Reduce  resources  required  to  develop,  field,  and  maintain  information 
systems. 

The  objectives  and  goals  of  the  DoD  are  being  implemented  throughout  the  development 
of  ACES  and  through  the  SDLC  process.  Compliance  with  these  goals  and  objectives 
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ensures  that  the  integrity  of  Air  Force  CE  data  is  preserved  under  a  single,  logical,  and 
relational  database  (14:6). 

ACES  also  adheres  to  Executive  Order  13011  by  utilizing  a  fully  automated 
information  system  for  all  CE  functions  (27:3).  The  Executive  Order,  approved  by 
President  Clinton  in  1996,  was  established  to  meet  requirements  set  forth  in  the  Paper 
Reduction  Act  of  1995  and  the  Information  Technology  Management  Reform  Act  of 
1996  (7:1).  The  Executive  Order  provides  an  opportunity  for  the  Federal  Government  to 
improve  the  way  they  acquire  and  manage  information  technology  (7:1). 

2.2.7  Process  for  ACES  Development  and  User  Initiated  Changes 

The  SSG  uses  a  version  of  the  SDLC  called  the  Systems  Engineering  Process 
(SEP)  to  design  and  maintain  the  ACES  system.  The  SEP  (see  Figure  3)  is  a  nine-step 
process  broken  down  into  three  separate  phases:  predevelopment,  development,  and 
post-development  (28:8).  The  SEP  was  developed  by  the  SSG  from  extensive  experience 
with  a  variety  of  software  and  system  development  efforts  (29:2).  The  ACES-PM 
module  is  currently  in  the  customer  support  step  within  the  post-development  phase 
where  maintenance  and  enhancements  occur  to  the  system  (28:8;  29:9). 
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2.2.8  Current  Status 


Currently,  five  of  the  eight  modules  have  entered  the  customer  support  phase  of 
the  SEP  and  are  undergoing  system  enhancements  (28:8;  49:53).  The  operations, 
environmental,  and  explosive  ordinance  modules  are  still  in  the  development  and  testing 
phase.  AFCESA’s  complete  schedule  and  status  of  all  eight  modules,  broken  down  by 
functionality,  is  shown  in  Figure  4  (current  as  of  November  2003). 
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Figure  4.  AFCESA’s  ACES  Implementation  Schedule  (49:53) 


2.3  Non-Appropriated  Funds  (NAF)  Projects 

The  ACES-PM  module  is  currently  in  the  SDLC  maintenance  phase,  where  many 
user  issues  dealing  with  business  rule  problems  have  been  brought  to  the  ACES-PM  IPT. 
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ACES-PM  is  extensively  utilized  in  programming  NAF  related  projects,  and  some  of  the 
issues  brought  forward  specifically  relate  to  NAF  project  programming  business  rules. 
The  project  programming  process  includes  defining  project  requirements,  creating 
priorities,  and  developing  initial  cost  estimations. 

2.3.1  Background 

NAF  project  funding  is  a  long  and  detailed  process  that  involves  many  separate 
organizations.  To  understand  how  NAF  programming  is  designed  to  work  in  ACES-PM, 
it  is  first  important  to  be  familiar  with  NAF  programming  in  general.  The  following 
sections  describe  the  programming  and  funding  process,  address  the  organizations  and 
regulations  responsible  for  funding  NAF  projects,  illustrate  how  ACES-PM  currently 
supports  NAF  project  programming,  and  discuss  business  rule  changes  proposed  by  the 
ACES-PM  IPT.  The  intent  of  this  discussion  is  to  stress  the  complexity  of  the  NAF 
programming  process  and  demonstrate  how  important  it  is  to  base  ACES-PM  on  the 
business  processes  associated  with  NAF  project  programming. 

2.3.2  Programming  and  Funding  Process 

The  first  and  most  important  task  of  programming  a  NAF  project  is  to  determine 
what  type  of  work  will  be  performed.  Work  classification,  along  with  facility  type,  is 
used  to  detennine  the  fund  source  and  program  avenue  (2:15-18;  23:6-13).  Detennining 
the  funding  source  is  not  always  obvious  and  AFI  32-1022,  Planning  and  Programming 
Nonappropriated  Funds  Facility  Construction  Projects,  provides  a  detailed  summary  of 
funding  sources  for  all  types  of  facilities  and  work  classifications  (23:6-13). 


31 


Programmers  are  required  to  review  these  fund  sources  to  ensure  that  a  project  can  be 
fully  or  partially  funded  using  NAF  before  inputting  it  into  ACES-PM  as  a  NAF  project. 
As  a  general  rule,  all  NAF  facility  construction  requires  NAF  funding  and 
maintenance/repair  projects  depend  on  the  facility  category  code  (3:22;  23:6-13).  After 
the  fund  source  is  determined,  a  record  is  created  for  the  project  in  ACES-PM. 

A  NAF  funding  source  depends  on  the  type  of  non-appropriated  fund 
instrumentality  (NAFI)  along  with  the  total  cost  of  the  project  (3:21).  NAF  are  “cash  and 
other  assets  that  NAFIs  generate  and  receive  from  sources  other  than  Congressional 
Appropriations”  (22:3).  NAFIs  are  agencies  of  the  government  that  provide  Morale, 
Welfare,  Recreation,  and  Services  (MWRS)  activities  and  programs  (22:3).  Revenue¬ 
generating  NAFIs  typically  belong  to  the  MWRS,  or  Services  community,  and  AAFES 
(3:4).  For  review,  NAFIs  are  generated  from  service  members’  patronage  at  AAFES 
outlets  and  other  MWRS  type  activities  (23:2).  For  Services-related  NAF  projects  under 
$200,000,  funds  come  from  NAF  dollars  generated  locally  (3:21).  If  the  project  is  over 
$200,000,  the  funds  come  from  the  Air  Force  Base  Capital  Improvement  Program  (3:21). 
For  AAFES  NAF  projects,  funds  come  from  local  revenues  for  projects  under  $500,000 
and  from  the  Long  Range  Capital  Improvement  Fund  for  projects  over  $500,000  (3:21). 
However,  bases  and  MAJCOMs  are  expected  to  fund  NAF  projects  that  are  less  than 
$200K,  and  projects  within  their  capability  (34).  Commissary  projects  are  funded  by 
surcharges  and  managed  by  the  Defense  Commissary  Agency  (DEC A)  (3:3). 

The  NAF  project  approval  authority  and  funding  source  is  based  on  the  type  of 
work  and  cost  estimate.  The  cost  estimate  for  approval  purposes  is  at  the  35  percent 
design  point  (23:29;  31:1).  For  all  NAF  maintenance  projects,  the  MAJCOM  has 
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unlimited  approval  authority  (23:30).  If  it  is  a  repair  project,  or  a  combination  of  repair 
and  maintenance,  the  MAJCOM  can  approve  the  project  up  to  $500,000  and  CE  at  Air 
Staff  for  over  $500,000  (23:30).  If  the  project  is  construction,  the  MAJCOM  approves 
projects  up  to  $500,000  and  Congress  for  over  $500,000  (23:29).  For  each  project, 
ACES-PM  should  adequately  display  the  approval  authority  and  the  types  of  funds  being 
used. 

A  NAF  project  cannot  exceed  10  percent  of  the  scope  and  25  percent  of  the  cost 
defined  at  the  35  percent  design  level  (23:3 1 ;  3 1 : 1 ;  32;  34).  When  a  project  scope 
exceeds  the  10  percent  threshold,  Congress  must  approve  the  increase  before  additional 
design  or  construction  funds  are  committed  (23:3 1).  When  a  project  exceeds  its  35 
percent  design  cost  estimate,  the  base  and  MAJCOM  are  responsible  for  funding  the  cost 
increase  arising  from  in-house  scope  changes  (34)  and  are  not  required  to  seek  any  higher 
approval  authority  (23:3 1).  However,  if  a  project  exceeds  the  cost  estimate  by  either  the 
25  percent  cost  threshold  or  $500,000,  approval  must  be  granted  from  the  Deputy 
Assistant  Secretary  of  the  Air  Force  (23:3 1).  Whenever  projects  exceed  the  approval 
thresholds,  all  programming  documents  must  be  updated  to  reflect  the  new  changes  and 
forwarded  to  the  appropriate  approval  authority.  ACES-PM  should  be  able  to  efficiently 
support  this  type  of  cost  and  scope  change  process. 

2.3.3  Documentation  Requirements 

The  amount  and  complexity  of  required  supporting  documentation  depends  on 
both  the  authority  approval  level  and  cost  of  the  project  (3:32;  23:38-39).  All  NAF 
projects,  independent  of  cost  and  approval  level,  require  at  least  an  AF  Form  332  (work 
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request)  or  DD  Form  1391,  and  an  Installation  Commander’s  Certificate  (CC)  (23:38). 
These  are  the  minimum  programming  documents  that  are  required  for  all  NAF  projects. 
An  AF  Form  332  can  only  be  used  for  projects  within  base-level  approval,  where  DD 
Form  1391s  are  used  for  all  other  approval  authorities  (23:38).  The  Installation  CC  is 
used  to  help  ensure  that  the  project  abides  by  all  programming  laws  and  policies;  it  is  also 
used  to  establish  a  measure  of  accountability  (23:38). 

Projects  within  the  installation  commander’s  approval  authority  need  the 
minimum  programming  documents  (23:38).  If  the  project  is  over  $50,000  for 
construction  or  $100,000  for  maintenance,  it  will  also  need  an  Internal  Needs  Validation 
Study  (INVS)  (24:4).  Projects  approved  at  the  MAJCOM  level  that  are  between 
$200,000  and  $500,000  require  a  DD  Fonn  1391  and  1391c,  INVS,  detailed  cost 
estimate,  single  line  drawings,  site  plan,  facility  deficiency  detail  data  sheet  (D3), 
installation  CC,  and  a  Needs  Assessment  Study  (NAS)  (23:38-39).  For  projects  greater 
than  $500,000,  three  additional  pieces  of  information  are  required:  joint  use  certificate, 
patronage  data,  and  Certificate  of  Environmental  Compliance  (23:38).  All  three  of  these 
requirements  are  included  on  the  DD  Form  1391c  (23:38). 

The  INVS  is  an  important  preliminary  document  that  must  be  accomplished  at  the 
installation  level  by  CE,  Services,  and  Financial  Management  personnel.  The  INVS  is  a 
critical  document;  it  informs  the  base  commander  of  the  project  need  and  is  the  baseline 
used  to  compete  against  other  projects  at  the  MAJCOM  level  (3;  37).  The  INVS  consists 
of  five  sections  (3;  38):  Part  1,  Project  Definition/Existing  Facility  Analysis;  Part  2, 
Existing  Facility  Operational  Analysis;  Part  3,  Financial  Considerations;  Part  4, 
Marketing/Customer/Other  Considerations;  and  Part  5,  Summary.  Parts  one  though  four 
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are  used  to  calculate  a  relative  need  score  that  is  used  to  help  justify  the  project  to  the 
installation  commander  and  MAJCOM  (3:1 1-12).  The  last  part  is  a  summary  of  the  four 
section’s  calculations  and  requires  the  installation  commander’s  signature  (38). 

Supporting  documentation  for  the  INVS  includes  a  detailed  cost  estimate 
normally  accomplished  through  the  Air  Force  Parametric  Cost  Estimating  System, 
Installation  Commander’s  Certificate,  and  selected  parts  of  the  DD  Form  1391  package 
(5;  23:38-39;  30:3).  All  these  documents  should  also  be  included  in  the  ACES-PM 
project  file.  They  are  all  important  preliminary  documents  that  capture  necessary  facts 
about  the  project.  They  need  to  be  filled  out  accurately  and  efficiently  included  with  the 
rest  of  the  ACES-PM  project  file. 

A  NAS  must  be  completed  for  all  projects  that  are  over  $200,000  (35).  The  NAS 
is  used  to  validate  the  market  demand  and  determine  the  most  appropriate  method  to  meet 
it  (3:48).  They  are  usually  conducted  by  a  civilian  contractor  if  the  project  is  over 
$500,000  or  by  AFSVA  in-house  personnel  if  the  project  cost  is  between  $200,000  and 
$500,000  (3:49).  A  site  visit  is  completed  along  with  a  detailed  functional  layout,  cost 
estimate,  and  requirements  documents.  The  NAS  is  used  by  the  AFSVA  Facilities  Panel 
to  prioritize  projects  for  the  35  percent  design  selection. 

2.3.4  Responsible  Organizations  and  Approval  Process 

Many  Air  Force  organizations  are  responsible  for  programming  and  funding  NAF 
projects.  After  the  initial  NAF  facility  project  requirements  are  determined  by  base-level 
personnel,  programmers  in  the  CE  organization  input  and  maintain  the  project 
information  in  ACES-PM.  The  project  file,  documents,  and  its  user  rights  are  then 
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transferred  back  and  forth  within  ACES-PM  between  several  responsible  organizations 
before  final  design  and  funding  is  achieved  (5).  Responsible  organizations  include: 
base-level  CE  and  Services  personnel;  MAJCOM  CE  and  Services  personnel;  and  higher 
headquarters  Air  Force  (Air  Staff)  CE  and  Services  personnel. 

2.3.4.1  Base-Level  Organizations 

Since  it  is  CE’s  responsibility  to  provide,  operate,  maintain,  and  restore  the 
installations  and  facilities  necessary  to  support  the  Air  Force  mission,  base-level  CE 
plays  an  important  role  in  the  NAF  programming  process  (2 1 :2).  They  identify  NAF 
facility  requirements,  with  help  from  functional  experts,  and  enter  the  relevant 
information  into  the  ACES-PM  database.  Base-level  CE  also  completes  their  required 
part  of  the  INVS,  Part  I.  Base-level  programmers  constantly  identify  new  requirements 
and  update  current  project  documentation  to  prepare  for  the  annual  NAF  project  call.  CE 
also  ensures  that  all  projects  are  in  conformance  with  the  installation  master  development 
plan  and  gains  approval  through  appropriate  base-level  decision-making  processes  (1 1:3). 

Once  a  project  is  approved,  personnel  from  base-level  CE  and  the  Services  work 
closely  with  the  architectural  and  engineering  firm  designing  the  project.  During  the 
design  process,  they  review  and  provide  comments  on  the  draft  NAS  (5;  35).  They  also 
attend  the  35,  65,  and  95  percent  project  review  meetings  and  provide  comments.  When 
the  design  effort  is  complete  and  the  project  is  recommended  for  funding,  CE  coordinates 
the  authority  to  advertise  and  award  the  construction  of  the  project  with  its  respective 
MAJCOM  (5).  Services  personnel  are  also  responsible  for  coordinating  the  completion 
of  the  INVS  along  with  filling  out  parts  II  and  IV  of  the  document.  Throughout  this 
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process,  it  is  essential  that  ACES-PM  be  designed  to  easily  capture  and  show  all  initial 
project  requirement  information  to  include  the  INVS  and  base-level  design  review 
comments. 

2.3.4.2  Major  Command  Organizations 

Once  the  installation  commander  concurs  with  the  NAF  facility  requirement,  the 
project  can  be  forwarded  to  the  appropriate  MAJCOM  during  the  annual  NAF  project 
call,  which  is  initiated  each  July  (3;  5).  It  is  the  MAJCOM ’s  responsibility  to  review 
each  project  folder  and  its  accompanied  documentation  for  accuracy  and  coordinate  with 
base-level  CE  to  correct  any  errors  (5).  MAJCOM  CE  personnel  will  review  the  existing 
facility  information  contained  in  the  INVS,  cost  estimate,  and  DD  Form  1391  package 
(5).  The  Services  personnel  will  review  the  functional,  operational,  and  marketing 
sections  of  the  INVS  (5).  Once  all  supporting  documentation  is  verified,  the  project  is 
prioritized  by  the  MAJCOM  NAF  council  and  submitted  to  the  NAF  Facilities  Panel  at 
the  Air  Force  higher  headquarters  level  (Air  Staff)  for  review  (5).  The  panel  consists  of 
one  CE  member  and  several  Services  representatives,  mostly  from  the  AFSVA  (5).  The 
MAJCOM  NAF  council  also  coordinates  with  MAJCOM  CE  personnel  prior  to 
submitting  the  prioritized  list  to  compete  for  design  funding  from  AFSVA  (5). 

Once  projects  are  prioritized  and  sent  to  AFSVA,  the  MAJCOM  does  not  get 
involved  in  the  process  again  until  the  NAS  has  been  completed  (3;  5;  35).  AFSVA 
provides  primary  oversight  during  the  accomplishment  of  the  NAS,  but  MAJCOM 
personnel  from  CE  and  Services  are  usually  invited  to  participate  (3;  5).  When  the  draft 
NAS  is  complete,  MAJCOM  personnel  have  14  days  to  review  it  and  provide  comments 
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to  AFSVA  (5;  30;  35).  The  NAS  is  finalized  by  the  NAS  contractor  or  AFSVA  in-house 
representatives  after  all  comments  have  been  incorporated  into  the  document  (5). 

The  MAJCOM’s  role  with  the  project  starts  again  after  AFSVA  reviews  the  NAS 
and  selects  projects  for  35  percent  design.  At  this  point,  MAJCOM  CE  personnel  will 
review  the  base’s  statement  of  work  and  assign  a  primary  project  manager,  who  will  be 
responsible  for  chairing  all  design  review  meetings.  They  will  also  review  the  accuracy 
of  the  DD  Form  1391  and  change  the  programmed  amount  to  match  the  NAS  cost 
estimate  (5).  This  step  is  important  since  the  cost  estimate  at  the  35  percent  design  stage 
will  be  locked  in  and  used  to  fund  the  project  if  it  is  selected  to  go  to  100  percent  design 
(30;  3 1).  The  MAJCOM  and  base-level  CE  personnel  will  then  coordinate  with  AFSVA 
to  select  the  method  of  design  and  award  the  design  to  a  NAF  architect-engineer  (AE) 
contractor  (5;  33).  MAJCOM  CE  personnel  are  also  required  to  attend  the  35,  65,  and  95 
percent  project  review  meetings  (5;  33). 

During  the  project  approval  process  described  above,  the  ACES-PM  project  file  is 
released  to  other  using  agencies  multiple  times.  Therefore,  CE  personnel  at  the 
MAJCOM  level  must  work  hard  to  ensure  the  integrity  of  the  information  contained  in 
the  ACES-PM  database  stays  intact.  Having  a  more  effective  way  to  track  the  project  in 
ACES-PM  would  tremendously  help  the  MAJCOM  personnel  manage  their  NAF  projects 
more  efficiently. 

2.3.4.3  Air  Force  Headquarters  Organizations 

There  are  several  organizations  responsible  for  NAF  projects  at  the  Air  Staff 
level.  On  the  civil  engineering  side,  the  Office  of  the  Civil  Engineer’s  Engineering 
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Division  is  responsible  to  “plan,  program,  advocate,  distribute  resources,  and  monitor 
execution  for  Air  Force  capital  improvement  programs”  (19).  In  this  capacity,  they  are 
the  focal  point  for  processing  approval  requests  and  reporting  all  NAF  projects  to  higher 
authorities  to  include  the  Secretary  of  the  Air  Force  (SAF),  Office  of  the  Secretary  of 
Defense  (OSD),  and  Congress  (3:5;  25:2).  They  are  directly  involved  in  approving 
projects  that  exceed  the  allowed  cost  and  scope  variation  limits  (23:3 1).  The  Engineering 
Division  issues  design  instructions  (DI)  for  projects  at  four  separate  points:  35  percent 
design,  100  percent  design,  authority  to  advertise,  and  authority  to  award  (5;  23:22).  A 
DI  is  required  to  obtain  the  grants  and  loans  necessary  to  start  project  design  and 
construction  (23:22).  To  track  each  DI  in  ACES-PM,  the  user  rights  are  transferred  back 
and  forth  between  the  MAJCOM  and  Air  Staff  level;  therefore,  it  is  imperative  that  the 
tracking  system  in  ACES-PM  allow  the  project  status  and  location  to  be  easily 
determined  at  any  time  during  the  funding  process. 

Within  the  Directorate  of  Services,  the  AFSVA  provides,  constructs,  and 
maintains  NAF  facilities  through  policy,  direction,  and  standards  (20;  24:2).  The  mission 
of  the  Directorate  of  Services  is  to  “contribute  to  readiness  and  improve  productivity 
through  programs  promoting  fitness,  esprit  de  corps,  and  quality  of  like  for  Air  Force 
People”  (20).  In  addition  to  reviewing  and  approving  the  prioritized  projects  lists 
produced  by  the  NAF  Facilities  Panel  (3:5),  the  Directorate  of  Services  also  approves 
scope  increases  greater  than  10  percent  (23:31). 

The  mission  of  AFSVA  is  to  “support  Air  Force  and  Services  leadership, 
MAJCOMs,  and  base  level  Service  Units  to  improve  the  quality  of  life  for  all  personnel 
and  their  families”  (15).  AFSVA  is  heavily  involved  in  the  NAF  project  funding  and 
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approval  process.  They  initiate  the  annual  call  to  the  MAJCOMs  for  projects,  review  and 
prepare  project  submittal  packages  for  NAF  Facilities  Panel  action,  certify  the  100 
percent  design,  issue  the  delivery  order,  approve  all  user  requested  changes,  and 
financially  close  out  projects  after  construction  is  complete  (5). 

The  NAF  Facilities  Panel  is  also  located  at  the  Air  Staff  level.  Chaired  by  the 
AFSVA  Deputy  Commander,  the  panel  includes  representatives  from  the  CE  and 
Services  communities  along  with  other  organizations  (3:47).  This  panel  reviews  and 
recommends  projects  for  the  NAS,  35  percent  design,  and  funding  (36). 

Each  of  the  responsible  organizations  described  above  is  an  important  part  in  the 
overall  programming  and  funding  process  for  NAF  projects.  The  full  Air  Force  NAF 
project  approval  process  is  shown  in  Figure  5. 
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2.3.5  NAF  Approval  Timeline 

The  NAF  programming  process  is  initiated  when  the  AFSVA  leadership  issues 
the  annual  project  call  down  through  the  Air  Staff  to  the  MAJCOMs  and  to  the 
installations  (37).  This  annual  project  call  specifies  all  the  project  information  and 
documents  that  are  required  to  compete  for  NAF  funding  (37).  The  actual  time  for  a 
project  that  requires  approval  from  Congress  is  around  two  years.  Part  of  the  reason  why 
the  approval  process  is  so  long  is  the  fact  that  the  AFSVA  reviews  and  recommends  NAF 
projects  during  three  separate  stages  of  the  process.  Another  reason  involves  the  long 
design  schedule  that  must  be  completed  in  order  for  Congress  to  approve  a  project  (5; 

39).  All  projects  that  the  NAF  Facilities  Panel  does  not  recommend  for  funding,  no 
matter  where  they  are  in  the  approval  process,  are  released  back  to  their  MAJCOM  and 
installation  to  compete  again  the  next  congressional  reporting/approval  cycle  (5).  The  2- 
year  reporting/approval  cycle  timeline  is  shown  in  Figure  6. 

In  the  summer  of  2003,  Services  at  Air  Staff  approved  many  changes  that  will 
eventually  affect  the  timeline  shown  in  Figure  6  along  with  other  NAF  issues  (4:1).  A 
couple  of  these  changes  include  changing  the  project  call  from  July  to  March,  and  having 
the  packages  due  to  AFSVA  in  September  instead  of  December  (4:2).  All  the  proposed 
changes  will  eventually  allow  NAF  projects  to  flow  through  the  approval  system  more 
efficiently  (4:1). 
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Figure  6.  NAF  Project  Approval  Timeline  (39) 


2.3.6  Regulations,  Guidance,  and  Standards 

Congress,  the  DoD,  and  the  Air  Force  stipulate  the  NAF  programming  process  in 
various  instructions,  regulations,  and  guidelines.  AFI  32-1022,  Planning  and 
Programming  Nonappropriated  Funds  Facility  Construction  Projects,  is  the  main 
guidance  for  programming  NAF  projects.  It  describes  how  to  “plan,  develop,  and  submit 
non-appropriated  fund  (NAF)  programs  to  approving  authorities”  (23:1).  The  guidance 
applies  to  the  following  agencies  from  the  Air  Force  to  the  installation  level  that  deal 
with  NAF  projects:  Army  and  Air  Force  Exchange  Service  (AAFES),  Air  Force 
Services,  Air  Force  Civil  Engineering  and  construction  managers,  the  guard  and  reserve, 
and  all  private  organizations  with  NAF  facility  contracts  (23:1).  However,  the  instruction 
does  not  apply  to  commissary  projects  funded  with  surcharges  since  those  fall  directly 
under  the  DoD  (23:1).  The  AFI  specifically  dictates  what  funding  source,  NAF  or  APF, 
is  used  for  each  type  of  facility  by  category  code,  funding  level,  and  the  work  perfonned 
(3:3;  23).  The  guidance  also  identifies  the  project  approval  authority,  the  laws  governing 
scope  changes  and  cost  variations,  and  the  documents  required  for  project  approval  and 
funding  support  (23:38). 

In  addition  to  AFI  32-1022,  which  applies  primarily  to  the  civil  engineering 
community,  there  are  four  other  key  documents.  AFI  65-106,  Appropriated  Fund 
Support  ofMWR  and  Nonappropriated  Fund  Instrumentalities  (NAFIs),  is  the  financial 
manager’s  guidance  on  NAF;  it  “provides  financial  guidance  on  using  APF  for  Air  Force 
Services  programs  and  NAFIs”  (13).  The  main  instructions  pertaining  to  the  customer 
are  AFI  34-2,  Managing  Nonappropriated  Funds',  AFI  34-105,  Programming  for 
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Nonappropriated  Fund  Facility  Requirements ;  and  AFI  34-201,  Use  of  Nonappropriated 
Funds. 

2.4  Current  NAF  Business  Rules  in  ACES-PM 

It  is  important  that  ACES-PM  NAF  business  rules  match  the  actual  NAF 
programming  and  funding  process  described  in  Section  2.3.  Development  of  the  ACES 
modules  and  improvement  to  the  system’s  existing  interface  is  an  on-going  process.  It 
has  been  acknowledged  that  many  problem  areas  have  arisen  within  the  NAF 
programming  business  rules  since  the  transition  from  IWIMS.  The  overarching  problem 
is  that  NAF  programming  is  forced  to  use  the  business  rules  that  were  designed  to 
support  operations/maintenance  (O&M)  and  military  construction  (MILCON)  projects 
(43).  To  address  these  problems,  a  users  meeting  was  held  with  the  goal  being  to  ensure 
that  ACES-PM  mirrors  the  entire  NAF  process  through  the  programming,  design,  project 
selection  and  awards  process  (5).  The  specific  problem  areas  and  recommendations 
identified  at  the  IPT  meeting  are  discussed  in  the  remainder  of  this  section,  broken  down 
by  the  individual  programming  tabs  located  in  ACES-PM.  A  complete  list  of  the 
proposed  NAF  business  rule  changes  is  shown  in  Appendix  A.  Until  the  proposed 
changes  take  place,  NAF  programming  is  being  accomplished  using  inefficient  business 
rules  that  were  designed  for  other  processes. 

From  the  results  of  the  IPT  meeting,  the  Project  Add,  Managers,  and  Milestones 
screens  require  changes.  The  Project  Add  screen  is  the  initial  programming  input  that 
controls  the  way  the  rest  of  the  project  file  and  business  rules  act.  It  is  essential  that  this 
screen  adequately  match  how  NAF  programming  is  accomplished.  Inserting  NAF  for 
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“program  type”  on  the  project  add  screen  leads  to  “fund  types”  and  “type  of  work” 
choices  that  are  not  related  to  NAF.  Programmers  are  currently  allowed  to  input  “O&M” 
in  the  “program  type”  field  and  put  “NAF”  under  the  “fund  subsource”  field.  This  leads 
to  data  integrity  problems  when  personnel  at  the  headquarters  and  Air  Force  level  try  to 
sort  and  query  projects  in  the  database.  This  option  needs  to  be  eliminated  since 
changing  “program  type”  to  NAF  is  the  most  efficient  way  to  categorize  the  type  of 
project.  The  only  change  required  for  the  Managers  screen  is  the  addition  of  a  AFSVA 
primary  role.  The  Milestones  screen  requires  several  additional  milestones  to  better 
support  NAF  programming;  it  does  not  include  dates  for  INVS  completion,  selection  for 
NAS,  35  percent  design,  or  100  percent  design. 

As  for  the  various  project  tabs  that  are  available,  the  Facility  Investment  Metric 
(FIM)  and  Environmental  tabs  do  not  have  business  rules  that  need  to  be  changed.  NAF 
projects  are  prioritized  for  funding  consideration  using  the  INVS  and  the  NAS  instead  of 
the  FIM.  The  Enviromnental  tab  is  the  same  for  O&M,  MILCON,  and  NAF  projects. 
Other  tabs  are  applicable  but  need  changes  to  improve  their  usefulness. 

The  Programming  tab  has  many  features  that  do  not  support  NAF  programming. 
The  option  to  check  the  “environmental”  or  “MILCON”  box  is  not  applicable  to  NAF 
projects.  The  “category  code,”  “CE/PBD,”  and  “Group”  data  fields  are  also  not 
applicable  and  only  confuse  the  initial  project  input  process.  The  tab  does  not  allow  for 
the  input  of  the  “Total  NAF  Investment,”  which  is  an  important  figure  in  determining 
funding  and  cost  thresholds,  and  does  not  allow  the  INVS  status  to  be  easily  displayed. 
Finally,  the  programming  tab  does  not  allow  for  designating  if  there  is  an  APF 
companion  project  or  identifying  the  approval  and  funding  authority. 
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The  Supplemental  tab  requires  several  revisions  to  better  assist  NAF  projects. 

The  “Base,”  “MAJCOM,”  and  “Air  Staff’  Program  check  boxes  do  not  have  a  date  box 
to  indicate  when  the  NAF  project  was  approved  by  the  indicated  organization.  Also, 
AFSVA  does  not  have  a  check  box  to  indicate  when  a  project  is  approved  by  them  and 
sent  to  Air  Staff. 

The  Design,  Contract  Management,  and  Funding  tabs  require  minor  changes.  On 
the  Design  tab,  the  “Date  of  Original  USAF  DI”  is  limiting  in  that  it  only  allows  for  one 
date.  There  should  be  options  to  also  include  a  DI  date  for  the  initial  project  selection  for 
design,  at  the  35  percent  design  stage,  and  at  the  100  percent  design  stage.  The  Contract 
Management  tab  does  not  support  the  25  percent  cost  and  10  percent  scope  thresholds.  It 
does  not  contain  a  safety  feature  that  prevents  modifications  exceeding  these  thresholds 
and  forces  the  programmer  to  obtain  new  approval  authority.  The  Funding  tab  needs 
more  appropriate  options  for  the  “Fund  Indicator”  field,  which  should  match  the 
responsible  organizations  and  force  the  project  user  rights  to  transfer  between  them.  For 
all  three  tabs,  the  “Total  NAF  Investment”  field  is  not  present. 

The  current  DD  Form  1391  must  be  changed  to  accurately  reflect  NAF 
programming  requirements.  “Unfunded  Furnishings,  Fixtures,  and  Equipment  (FFE)” 
and  “Other  Appropriations  (Non-add)”  are  critical  to  NAF  funding  and  should  be 
included  in  Block  9  of  the  form.  The  “Total  NAF  Investment”  should  also  be  included  in 
Block  9.  Companion  projects  should  be  identified  on  the  form  with  their  project  title, 
project  number,  and  estimated  cost. 

There  are  general  areas  of  ACES-PM  that  also  need  to  be  changed  to  better 
support  NAF  projects.  Since  the  project  file  changes  user  rights  many  times  during  the 
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funding  process,  a  system  needs  to  be  put  in  place  that  easily  notifies  responsible 
organizations  that  project  rights  have  been  transferred  to  them  and  an  action  is  required 
on  their  part.  The  INVS  and  other  NAF-specific  documents  should  be  able  to  be 
electronically  attached  within  the  project  file.  When  the  DD  Form  1391  gets  updated  to 
reflect  the  changes  from  the  NAS,  a  method  has  to  be  developed  to  efficiently  allow 
personnel  at  all  levels  to  provide  their  input  and  approval. 

2.5  Chapter  Summary 

The  literature  review  was  crucial  in  understanding  the  basic  concepts  of  ACES 
and  NAF  programming.  The  background  on  ACES  PM,  its  responsible  organizations, 
and  status  helped  comprehend  how  the  AIS  works  in  general  and  how  to  implement 
changes  to  the  system.  The  review  of  NAF  programming  was  required  to  gain  more 
knowledge  in  how  the  funding  process  is  supposed  to  be  completed.  Finally,  a  look  at  the 
current  ACES-PM  business  rules  illustrated  how  the  current  ACES-PM  module  does  not 
support  the  NAF  programming  process. 
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3.  METHODOLOGY 


This  chapter  explains  the  methodology  used  to  answer  the  questions  proposed  in 
this  research  effort.  It  not  only  outlines  the  overall  research  design,  but  also  discusses  the 
reliability  and  validity  of  the  survey-based  research.  After  explaining  how  the  survey 
was  constructed  and  administered,  the  chapter  briefly  discusses  the  descriptive  statistics 
used  to  answer  the  research  questions.  Descriptive  statistics  were  used  to  generalize  the 
responses  to  Likert  Scale  questions  pertaining  to  proposed  changes  to  the  Automated 
Civil  Engineer  System  Project  Management  module  (ACES-PM)  non-appropriated  funds 
(NAF)  business  rules.  The  survey  questions  also  addressed  current  programming 
processes  that  might  hinder  implementation  of  the  proposed  changes  and  training  issues. 

3.1  Survey  Design  and  Administration 

3.1.1  Survey  Design 

A  cross-sectional,  quantitative  web-based  survey  was  created  as  the  instrument  to 
answer  the  research  questions  (42:155).  Since  the  survey  addressed  precise  questions 
related  only  to  NAF  programming  in  ACES-PM,  it  was  specifically  designed  for  this 
research.  A  survey  was  chosen  as  the  research  instrument  because  it  provided  a 
quantitative  description  of  the  opinions,  attitudes,  and  ideas  of  the  sample  under 
investigation  (8:153).  The  Likert  Scale  portion  of  the  survey  allowed  the  data  to  be 
analyzed  quantitatively. 

The  survey  was  developed  to  address  specific  changes  to  the  ACES-PM  NAF 
business  rules,  current  NAF  programming  procedures,  and  NAF  training  issues.  The 
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questions  were  broken  into  three  sections  based  on  the  type  of  question.  Demographic 
(D)  questions  were  fill  in  the  blank,  General  (G)  questions  used  a  7-point  Likert  Scale, 
and  Multi  (M)  questions  were  multiple-choice/fill  in  the  blank.  For  the  Likert  Scale 
questions,  “1”  represented  “Completely  Disagree”  and  “7”  represented  “Completely 
Agree.”  The  Likert  Scale  also  allowed  each  participant  to  answer  “NA”  if  they  did  not 
have  an  opinion  for  the  question.  The  complete  Likert  Scale  is  shown  in  Appendix  B  at 
the  beginning  of  the  G  section. 

The  majority  of  the  survey  questions,  G6  through  G23,  were  created  from  the 
results  of  an  integrated  process  team  (IPT)  meeting  held  on  7-9  October  2003.  During 
this  IPT,  ACES-PM  experts  met  to  detennine  ways  the  current  NAF  business  rules  can  be 
improved  to  better  match  the  NAF  programming  process.  The  first  day  was  spent 
mapping  out  the  existing  NAF  programming  process  for  all  the  responsible  organizations. 
The  last  two  days  were  used  to  review  the  existing  ACES-PM  business  rules  and  identify 
how  they  currently  supported  the  NAF  programming  process.  For  each  ACES-PM 
programming  tab  and  screen,  recommendations  for  specific  changes  to  the  automated 
information  system  (AIS)  were  made  to  better  support  NAF  programming.  The  results  of 
the  meeting  are  the  set  of  proposed  programming  business  rule  changes  that  are  listed  in 
section  G  of  the  survey.  Questions  G8,  9,  10,  11,  and  17  address  potential 
implementation  problems  due  to  current  programming  procedures.  These  questions  were 
also  created  from  the  results  of  the  IPT  meeting. 

Recall  that  the  complete  list  of  the  proposed  NAF  business  rule  changes  is  shown 
in  Appendix  A.  Note  that  the  recommended  changes  with  an  asterisk  (*)  are  changes  or 
issues  that  are  specifically  addressed  in  the  survey  questions.  These  proposed  changes 
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were  concentrated  on  since  they  were  the  most  important  in  terms  of  getting  a  base  level 
perspective  on  their  effectiveness. 

The  questions  that  were  not  developed  directly  from  the  IPT  dealt  with  NAF 
training  issues.  Questions  G1  through  G5  were  created  to  measure  the  knowledge  and 
training  level  of  personnel  regarding  NAF  programming  and  ACES-PM.  They  also 
investigated  programmers’  access  to  the  proper  resources  to  get  questions  answered. 

The  majority  of  questions  provided  a  limited  comment  section  in  which 
respondents  could  address  issues  that  were  not  contained  in  the  actual  survey  question. 
Likert  Scale  questions  had  a  comment  section  that  could  accommodate  up  to  250 
characters,  while  the  final  question,  M3  3,  was  unlimited  since  it  addressed  the  whole 
process  in  general.  The  comment  sections  were  optional  for  all  questions.  Unfortunately, 
none  of  the  comment  sections  were  used  by  the  respondents,  thereby  making  content 
analysis  impossible  for  this  research. 

The  research  questions  that  were  introduced  in  Chapter  1 ,  along  with  their 
purpose,  method  of  statistical  analysis,  and  corresponding  survey  questions  are  explained 
in  Table  4.  It  is  often  useful  to  show  how  each  research  question  is  related  to  the  specific 
questions  on  the  instrument  (8:159).  The  survey  and  informational  cover  letter  are  shown 
in  Appendix  B. 
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Table  4.  Research  Questions  (Purpose,  Survey  Questions,  and  Analysis) 


RESEARCH  QUESTION 

PURPOSE 

SURVEY 

QUESTIONS 

ANALYSIS 

1 .  How  well  accepted  is  each  of  the 
proposed  ACES-PM  NAF  project 
programming  business  rule  changes 
in  terms  of  the  Air  Force  base  level 
programming  community? 

This  tested  how  programmers  rank  each 
of  the  proposed  changes  to  ACES-PM. 
The  ratings  gave  the  IPT  a  measure  of 
how  well  accepted  the  changes  will  be. 

G6,  G7,  G12,  G13, 
G14,  G15,  G16,  G18, 
G19,  G20,  G21,  G22, 
G23 

Descriptive  Statistics 
using  Likert  Scale 
questions  that  pertain 
to  actual  changes. 

2.  How  well  does  the  current  base 
level  ACES-PM  programming 
process  support  the  new  ACES-PM 
NAF  business  rule  changes? 

This  gives  the  IPT  insight  into  whether 
the  proposed  changes  might  not  be  as 
efficient  as  predicted  because  of  current 
base  level  programming  procedures. 

G8,  G9,  G10,  Gil, 
G17,  Ml,  M2,  M3, 

M4 

Descriptive  Statistics 
of  Likert  Scale 
questions  that  pertain 
to  training, 
knowledge,  and 
resources. 

3.  How  well  trained  are  current  base 
level  programmers  in  NAF 
programming  and  NAF 
programming  in  ACES-PM? 

This  gives  the  IPT  insight  into  how  well 
trained  current  programmers  are  with 
respect  to  NAF  programming  and  NAF 
programming  in  ACES-PM. 

G1,G2,  G3,  G4,  G5 

Descriptive  Statistics 
of  Likert  Scale 
questions  that  pertain 
to  current 
programming 
procedures. 

Standard  demographic  information  about  the  respondents  was  collected  with 
survey  questions  Dl,  D2,  and  D3.  Although  these  questions  did  not  directly  contribute  to 
the  objectives  of  this  research,  they  were  useful  in  identifying  trends.  The  fourth 
demographic  question  (D4)  was  used  to  filter  out  participants  who  have  not  had  any  NAF 
programming  experience  in  ACES-PM.  Question  M5  was  an  opened-ended  question  that 
allowed  respondents  to  identify  and  discuss  NAF  programming  issues  that  were  not 
included  in  the  survey  questions.  Flowever,  none  of  the  respondents  answered  this 
question. 

3.1.2  Survey  Population  and  Sample 

The  majority  of  the  survey  questions  come  from  the  recommendations  made  by 
members  of  the  IPT.  Since  the  IPT  was  comprised  of  personnel  from  of  the  headquarters 
and  Air  Force  levels,  the  questions  are  biased  towards  their  point  of  view.  To  obtain  the 
opinions  and  recommendations  of  base-level  personnel,  the  population  for  this  survey 
was  considered  to  be  all  base-level  programmers.  However,  since  ACES-PM  was  first 
implemented  less  than  3  years  ago,  the  survey  population  was  further  limited  to  base- 
level  CE  programmers  who  had  recent  experience  in  programming  NAF  projects  in 
ACES-PM.  Question  D4  of  the  survey  (Do  you  have  experience  inputting  NAF  projects 
into  ACES-PM?)  attempted  to  filter  out  individuals  with  no  recent  experience;  if 
respondents  answer  “No”  to  the  question,  the  web  site  automatically  finished  the  survey. 
When  a  sample  is  chosen  for  a  particular  purpose  in  this  manner,  it  is  called  purposive 
sampling  (42:219). 
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In  addition  to  identifying  the  survey  population,  it  is  also  important  to  detennine 
the  approximate  size  of  the  population  and  ensure  an  adequate  sample  size  for  purposes 
of  analysis  (8:157;  46:316).  The  larger  the  sample  the  better;  however,  this  may  not 
always  be  practical  (42:221).  One  advantage  of  having  a  distinct  and  homogenous 
population,  like  base-level  programmers  with  ACES-PM  NAF  programming  experience, 
is  that  is  cuts  down  on  the  required  sample  size  and  allows  for  a  more  precise  calculation 
of  the  population  (42:221).  In  order  to  come  up  with  a  rough  number  of  base-level 
programmers  that  should  be  sampled,  and  the  desired  number  of  responses,  a  rough 
estimate  of  the  total  population  needs  to  be  determined. 

ACES-PM  was  fully  implemented  in  late  2001,  thereby  providing  two  years 
worth  of  NAF  project  programming  experience  in  ACES-PM  (49:53).  Assuming  that 
there  is  an  average  of  two  programmers  assigned  to  a  base-level  programming  office,  and 
they  only  spend  about  a  year  in  the  office,  there  would  be  about  four  programmers  from 
each  base  that  have  had  recent  ACES-PM  NAF  programming  experience.  Since  there  are 
84  bases  with  CE  organizations,  it  was  estimated  that  the  population  size  was  336.  This 
is  a  conservative  number  since  some  base  level  programmers  spend  more  than  one  year  in 
that  office,  and  some  bases  have  less  than  two  people  in  their  programming  offices.  For 
a  population  in  the  range  of  400  to  600,  at  least  50  percent  of  the  population  should  be 
sampled  (42:221).  Thus,  the  goal  is  to  have  168  survey  responses  for  the  statistical 
analysis.  This  is  an  optimistic  number  considering  the  survey  will  have  to  achieve  a  50 
percent  response  rate  assuming  it  reaches  the  whole  population. 
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3.1.3  Survey  Administration 

The  survey  was  administered  through  the  web  for  three  primary  reasons.  First,  in- 
house  expertise  was  available  to  support  the  development  of  a  web-based  survey. 

Second,  the  speed  of  delivering  the  survey  to  the  sample  population  and  obtaining 
responses  was  very  quick.  Third,  a  web-based  survey  allowed  for  automatic  compilation 
of  the  data;  this  eliminated  both  the  time  required  for  and  the  human  errors  associated 
with  manually  transferring  survey  responses  to  Excel  spreadsheets.  The  definition  of  the 
population  and  the  method  for  delivering  the  survey  also  eliminated  the  chance  of 
sampling  bias,  which  is  a  bias  that  interferes  with  how  the  sample  population  is  selected 
(42:222). 

The  web-based  survey  could  not  be  distributed  to  base-level  programmers  before 
a  Survey  Control  Number  (SCN)  was  issued  from  the  Chief  of  the  Air  Force  Survey 
Branch  at  the  Air  Force  Personnel  Center.  The  survey  also  had  to  be  approved  by  the 
local  Institutional  Review  Board  before  being  disseminated  to  the  participants.  Since  the 
survey  guaranteed  total  anonymity  of  each  participant  and  the  responses  did  not  put  them 
at  risk,  the  survey  was  granted  an  exemption  under  the  Code  of  Federal  Regulations,  title 
32,  part  219,  section  101,  paragraph  (b)  (2).  The  Institutional  Review  Board  exemption 
letter  is  shown  in  Appendix  C. 

Once  the  SCN  was  received,  the  web  survey  link  was  emailed  to  all  programming 
offices  at  the  headquarters  level  on  10  December  2003  with  an  explanation  of  the 
survey’s  purpose  and  instructions  to  distribute  it  to  base-level  personnel  (see  Appendix  D 
for  a  copy  of  the  email).  By  asking  the  headquarters  organizations  to  distribute  the 
survey,  it  was  hoped  that  more  credibility  would  be  provided  to  the  survey.  However, 
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only  eight  responses  to  the  survey  were  received  within  the  first  month  of  distributing  the 
survey  in  this  manner.  At  that  point,  the  following  alternatives  were  used.  The 
researcher  sent  the  survey  directly  to  base-level  programmers  using  the  Air  Force  global 
address  list.  Additionally,  about  80  programmers  taking  a  satellite  course  in  the  Civil 
Engineering  and  Services  School  (CESS)  were  administered  the  survey;  they  were  also 
provided  the  web  survey  link  with  instructions  to  forward  it  to  fellow  programmers.  The 
research  sponsor  and  IPT  chairpersons  also  resent  the  survey  to  all  headquarters  units 
stressing  the  importance  of  the  research  goals.  The  alternate  distribution  methods  helped 
improve  the  response  rate,  which  led  to  a  more  rigorous  statistical  analysis  of  the  data. 

3.2  Research  Reliability  and  Validity 
3.2.1  Measurement  Instrument  Reliability 

Reliability  is  the  extent  to  which  a  survey  instrument  consistently  yields  results 
when  the  characteristic  or  items  being  evaluated  have  not  changed  (42:99).  The  more 
reliable  the  instrument,  the  better  the  chance  to  draw  appropriate  conclusions  from  the 
survey  data  collected  (42:100).  There  are  four  forms  of  reliability:  Interrater  reliability, 
equivalent  forms  reliability,  test-retest  reliability,  and  internal  consistency  reliability 
(42:99).  Interrater  reliability  is  the  extent  to  which  the  responses  of  two  or  more  people 
yield  the  same  results  (42:99).  Equivalent  forms  reliability  is  the  degree  that  two 
different  versions  of  the  instrument  produce  similar  outcomes  (42:99).  Test-retest 
reliability  is  the  extent  the  instrument  produces  the  identical  results  on  two  different 
occasions  (42:99).  These  three  forms  of  reliability  measure  the  external  reliability,  or 
consistency  of  an  instrument.  However,  due  to  the  nature  of  the  instrument  being 
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constructed  solely  for  this  research,  and  time  constraints,  these  three  forms  of  external 
reliability  could  not  be  measured.  The  internal  consistency  form  of  reliability  is  the 
extent  to  which  all  the  items  yield  similar  results  (42:99).  Internal  consistency  of  Likert 
Scale  survey  questions  can  efficiently  be  measured  using  Cronbach’s  Alpha.  However, 
to  use  Cronbach’s  Alpha,  multiple  items  need  to  exist  for  each  construct  under 
investigation.  Since  each  survey  question  involved  a  separate  construct,  or  a  specific 
change  to  ACES-PM  NAF  project  programming,  the  internal  consistency  reliability  of 
each  construct  could  not  be  detennined. 

3.2.2  Measurement  Instrument  Validity 

It  is  important  to  have  validity  in  the  measurement  instrument,  which  is  defined  as 
“the  extent  to  which  the  instrument  measures  what  it  is  supposed  to  measure”  (42:  98). 
There  are  four  forms  of  measurement  instrument  validity:  Face,  Content,  Criterion,  and 
Construct  validity  (42:98).  Face  validity  is  the  extent  to  which  an  instrument  looks  like  it 
is  measuring  the  correct  characteristics  (42:  98).  Face  validity  helps  ensure  the 
cooperation  of  the  survey  participants,  but  it  is  not  an  exact  measure  of  validity  (42:98). 
Participants  will  be  more  obliged  to  partake  in  the  survey  if,  on  the  surface,  they  trust 
what  it  is  asking  (42:98).  Each  of  the  survey  questions  was  reviewed  to  ensure  that  they 
appeared  to  be  measuring  the  specific  ACES-PM  NAF  programming  issue  in  question. 

Content  Validity  is  the  extent  to  which  an  instrument  is  an  accurate  representation 
of  the  content  area  being  measured  (42:  98).  One  way  to  measure  content  validity  is  to 
have  the  instrument  measured  by  experts  in  the  field  with  a  pilot  or  field  test  (8: 158;  42: 
98).  To  meet  this  requirement,  the  survey  was  sent  to  the  Civil  Engineering  School’s 
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NAF  Programming  Instructor  before  being  released  to  base-level  programmers.  Most  of 
her  comments  and  recommendations  were  incorporated  into  the  final  instrument 
revisions.  The  survey  was  also  sent  to  fellow  classmates  who  have  had  NAF 
programming  experience  within  ACES-PM  during  the  previous  assignment.  The 
feedback  received  from  both  sources  helped  ensure  that  the  instrument  was  indeed 
measuring  the  specified  content  area.  Along  with  establishing  content  validity,  this  step 
also  helped  improve  the  questions,  scales,  and  format  of  the  web-based  survey.  It  also 
helped  improve  the  accuracy  and  usability  of  the  instrument.  Additionally,  it  helped 
correct  many  functional  problems  and  errors  associated  with  web-based  surveys  that 
could  have  affected  the  validity  and  quantity  of  the  survey  responses. 

Criterion  validity  is  the  extent  that  the  results  of  the  instrument  compare  with 
related  instruments  that  measure  the  same  principle  (42:98).  Since  the  survey  was 
constructed  from  issues  specifically  related  to  proposed  NAF  business  rule  changes,  and 
there  are  no  other  instruments  to  measure  this,  criterion  validity  cannot  be  determined. 

Construct  validity  is  the  extent  that  the  instrument  reflects  the  characteristics,  or 
constructs,  that  cannot  be  directly  measured  (42:98).  Each  question  pertaining  to  the 
proposed  NAF  changes  or  current  programming  procedures  deals  with  a  specific 
construct.  These  questions  were  individually  reviewed  to  ensure  they  focused  directly  on 
the  proposed  construct.  The  training  related  questions  were  reviewed  to  make  certain 
they  only  measured  constructs  related  to  NAF  training  issues.  Careful  and  precise 
definition  of  all  the  constructs  helped  ensure  that  construct  validity  was  met. 
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3.2.3  Research  Validity 

Internal  and  external  validity  are  used  to  measure  the  validity  of  a  research  project 
as  a  whole  (42:103).  Internal  validity  is  the  extent  to  which  the  research  study  can  draw 
accurate  conclusions  about  the  relationships  within  the  data  (42:  103-104);  it  must  be 
confirmed  before  external  validity  is  investigated  (42:103).  It  is  important  that  the 
researcher  has  confidence  that  correct  conclusions  can  be  drawn  from  the  data  where  the 
results  can  be  explained  by  known  causes  (42:105).  In  order  to  check  for  internal 
validity,  the  researcher  must  eliminate  all  other  possible  explanations  for  the  observed 
results  (42:104).  Even  though  this  research  does  not  have  cause-effect  relationships, 
internal  validity  is  still  important  (42:105).  The  survey  questions  were  reviewed  to 
ensure  that  each  construct  being  measured  could  not  be  confused  with  other  constructs. 
For  example,  the  survey  questions  regarding  the  proposed  NAF  programming  changes 
were  investigated  to  make  sure  that  they  did  not  cover  other  ACES-PM  issues  that  did  not 
involve  NAF  programming. 

External  validity  is  the  extent  to  which  the  research  results  can  be  generalized  to 
other  situations  beyond  the  actual  research  (42:105).  Since  this  research  is  only  specific 
to  NAF  programming  within  ACES-PM,  and  cannot  be  applied  to  other  situations, 
external  validity  is  not  required.  The  results  of  this  research  will  only  pertain  to  base- 
level  programmers  who  have  experience  with  NAF  projects  in  ACES-PM. 

3.3  Statistical  Analysis 

Descriptive  statistics  measure  what  data  look  like  in  general  to  include  their 
central  tendency,  frequency,  and  variation  (42:259).  Descriptive  statistics  will  be  used 
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on  the  Likert  Scale  and  multiple  choice  questions  to  describe  the  results  of  the  research 
questions.  Since  Likert  Scale  data  are  ordinal  in  nature,  the  median  instead  of  the  mean 
will  be  used  to  describe  the  central  tendency  for  each  question  and  the  inter  quartile  range 
(IQR)  will  be  used  in  lieu  of  the  standard  deviation  (47).  The  IQR  is  a  measure  of  the 
data  spread  and  is  calculated  by  taking  the  difference  between  the  75th  percentile  and  the 
25th  percentile  of  the  data  set. 

3.4  Chapter  Summary 

This  chapter  established  the  procedures  of  how  the  research  instrument  was 
selected  along  with  how  the  survey  was  develop,  administered,  and  analyzed.  The 
results  of  the  survey  responses  were  used  to  answer  all  three  research  questions. 
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4.  RESEARCH  RESULTS 


This  chapter  summarizes  the  results  of  the  descriptive  statistics  of  the  survey 
responses.  The  survey  results  are  shown  first  to  include  the  sample  size  and 
demographic  infonnation  for  the  respondents.  Descriptive  statistics  were  then  used  to 
answer  the  research  questions  posed  in  Chapter  1 . 

4.1  Survey  Response  Results 

The  survey  was  sent  electronically  using  the  several  different  techniques 
discussed  in  Chapter  3.  There  were  only  35  responses  to  the  survey.  Using  the  estimated 
population  size  of  336,  this  represents  a  10  percent  response  rate,  much  less  than  the 
original  goal  of  50  percent.  The  lack  of  participation  in  the  survey  resulted  in  the 
elimination  of  three  research  questions  that  would  have  been  based  on  inferential 
statistics  and  content  analysis.  The  implications  of  the  low  response  rate  are  addressed  in 
Chapter  5.  The  demographics  pertaining  to  the  survey  respondents  are  shown  in  Tables  5 
and  6. 


Table  5.  Survey  Responses  by  MAJCOM 


MAJCOM 

ACC 

AETC 

AFMC 

AFSOC 

AFSPC 

AMC 

PACAF 

USAFE 

TOTAL 

#  RESPONSES 

13 

5 

9 

0 

1 

1 

3 

3 

35 

PERCENT 

37.1% 

14.3% 

25.7% 

0.0% 

2.9% 

2.9% 

8.6% 

8.6% 

100.0% 
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Table  6.  Survey  Responses  by  Rank/Grade 


RANK/GRADE 

2LT 

1LT 

CAPT 

GS 

OTHER 

TOTAL 

#  RESPONSES 

4 

6 

3 

16 

6 

35 

PERCENT 

11.4% 

17.1% 

8.6% 

45.7% 

17.1% 

100.0% 

AVERAGE 

PROGRAMMING 

EXPERIENCE 

(yrs) 

0.9 

1.8 

1.7 

7.3 

9.8 

5.6 

AVERAGE  NAF 
PROJECTS 
PROGRAMMED 

5.8 

12.7 

9.0 

42.8 

16.0 

25.9 

As  shown  in  Table  5,  the  majority  of  responses  came  from  two  major  commands 
(MAJCOMs):  Air  Combat  Command  and  Air  Force  Materiel  Command.  Three 
MAJCOMs  had  only  one  or  no  responses  each:  Air  Force  Special  Operations  Command, 
Air  Mobility  Command,  and  Air  Force  Space  Command.  It  can  be  assumed  that  some 
MAJCOMs  did  not  put  a  lot  of  emphasis  on  having  their  base-level  programming  offices 
participate  in  the  survey.  The  subsequent  methods  used  to  disseminate  the  survey  also 
could  have  affected  the  amount  of  responses  from  each  MAJCOM  since  the  survey  link 
was  forwarded  to  personnel  by  means  other  than  MAJCOM  affiliation. 

Table  6  shows  that  almost  half  the  respondents  were  general  schedule  (GS) 
civilian  employees.  The  higher  amount  of  GS  responses  might  be  because  they  are  the 
most  involved  in  non-appropriated  funds  (NAF)  programming  in  the  Automated  Civil 
Engineer  System  Project  Management  Module  (ACES-PM)  and  hold  pennanent 
programming  positions.  The  table  also  demonstrates  that  most  of  the  NAF  programming 
experience  resides  in  the  GS  and  Other  employee  categories.  It  is  assumed  that  most  of 
the  respondents  in  the  “Other”  category  represented  independent  contractors.  The 
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amount  of  projects  programmed  is  also  heavily  weighted  towards  the  GS  employees  for 
the  reason  mentioned  earlier.  A  student  t-test  of  independent  samples  was  attempted  to 
determine  whether  there  was  a  significant  difference  of  the  means  of  projects 
programmed  between  civilians  and  military.  However,  the  distribution  of  projects 
programmed  was  not  normal  and  the  independent  samples  did  not  have  a  common 
variance  which  has  to  be  met  before  a  t-test  can  be  performed. 

4.2  Research  Question  1 

Recall  that  the  first  research  question  was:  “How  well  accepted  is  each  of  the 
proposed  ACES-PM  NAF  project  programming  business  rule  changes  in  terms  of  the  Air 
Force  base-level  programming  community?”  The  list  of  the  integrated  process  team’s 
(IPT’s)  proposed  business  rule  changes  is  shown  in  Appendix  A.  Note  that  only  the 
proposed  changes  labeled  with  an  asterisk  (*)  were  addressed  in  the  survey  questions 
pertaining  to  research  question  1.  There  were  15  survey  questions  used  to  answer  this 
question.  Each  question  addressed  a  specific  change  to  the  NAF  programming 
procedures  that  was  recommended  by  the  ACES-PM  IPT.  The  median  and  interquartile 
range  (IQR)  for  each  of  the  Likert  Scale  questions  pertaining  to  research  question  1  were 
calculated  and  are  shown  in  Table  7.  For  the  specific  content  of  each  question,  the  reader 
is  referred  to  Appendix  B. 
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Table  7.  Descriptive  Statistics  for  Likert  Scale  Questions  (Research  Question  1) 


SURVEY 

QUESTION 

MEDIAN 

IQR 

G6 

5 

2 

G7 

5 

2 

G12 

6 

1 

G13 

6 

1 

G14 

5 

1.5 

G15 

5 

1.5 

G16 

5 

2.0 

G18 

5 

2 

G19 

4 

2.5 

G20 

6 

2 

G21A 

5 

1.5 

G21B 

5 

1 

G21C 

5 

2 

G22 

5 

1.5 

G23 

5 

2 

Every  survey  question  shown  in  Table  7  except  G19  yielded  a  median  score  of  5 
or  6,  which  indicated  base  level  programmers  “agree”  or  “strongly  agree”  that  the 
proposed  changes  will  be  beneficial  in  improving  NAF  programming  in  ACES-PM.  The 
three  most  agreed  upon  questions  were  G12,  G13,  and  G20,  all  with  a  median  score  of  6. 
Questions  G12  and  G13  suggested  mandatory  check  boxes  for  companion  projects  and  a 
field  to  enter  the  associated  project  number  and  cost  estimate  (or  programmed  amount). 
Both  questions  had  an  IQR  of  1,  indicating  that  most  programmers  strongly  agree  to 
some  degree  that  companion  projects  have  to  be  better  supported  within  ACES-PM. 
Question  G20  suggested  that  a  mandatory  check  box  should  be  added  to  ensure  the 
project  has  been  approved  by  the  installation  Facility  Board  before  being  input  as  a  valid 
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project.  The  IQR  of  2  indicates  that  most  programmers  agree  to  some  amount  that 
mandating  Facility  Board  approval  should  be  incorporated  in  ACES-PM. 

The  only  question  that  yielded  a  median  score  of  4  was  question  G19.  This  result 
indicates  that  base-level  programmers  are  generally  “undecided”  on  whether  projects  that 
are  not  approved  for  the  next  part  of  the  approval  process  (Needs  Assessment  Study 
(NAS),  35  &  95  percent  designs)  should  be  released  back  to  the  base.  However,  the 
question  also  had  the  largest  IQR,  indicating  that  responses  were  dramatically  split  on 
whether  this  option  would  be  beneficial  to  NAF  programming. 

The  remaining  questions  had  median  scores  of  5,  which  meant  that  the 
respondents  “agreed”  with  the  proposed  changes.  Questions  G6  and  G7  were  concerned 
with  effective  tracking  of  a  project’s  status.  Question  G6  suggested  the  use  of  automated 
email  notifications  to  notify  users  of  changes  in  project  status  and/or  that  an  action  is 
required  by  the  using  agency.  Question  G7  discussed  the  use  of  a  history  scroll  down 
field  in  the  programming  tab  that  lets  users  review  the  history  of  a  project.  Both 
questions  had  an  IQR  of  2,  signifying  that  the  majority  of  programmers  do  not  disagree  to 
any  extent  with  these  new  processes  being  beneficial  to  NAF  programming. 

The  survey  respondents  “agreed”  that  the  proposed  changes  outlined  in  questions 
G14,  G15,  and  G16  would  be  beneficial  to  ACES-PM.  All  three  questions  address  the 
addition  of  more  Internal  Needs  Validation  Study  (INVS)  information  to  the  project  file. 
Question  G14  recommended  adding  a  “pre-INVS”  status  block  that  lets  users  know 
whether  a  project  is  important  enough  to  warrant  the  start  of  an  INVS.  Question  G15 
covered  the  addition  of  a  data  field  that  allows  programmers  to  know  the  INVS  start  and 
finish  dates.  Question  G16  addressed  the  possibility  of  making  it  mandatory  to  attach  the 
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entire  INVS  document  electronically  to  the  project  file.  The  IQR  for  the  questions  was 
1.5,  1.5,  and  2,  respectively.  These  numbers  indicate  that  the  majority  of  responses  were 
spread  between  “undecided”  and  “strongly  agree.” 

Question  G18  asked  the  programmers  if  they  think  it  would  be  beneficial  to  allow 
Services  personnel  to  have  read-only  rights  to  ACES-PM  to  view  NAF  projects.  The 
survey  respondents  “agree”  that  Services  personnel  should  have  access  to  the  ACES-PM 
program.  According  to  the  IQR,  the  majority  of  programmer  thought  is  was  a  good  idea 
to  grant  this  read-only  option  to  Services. 

Question  G21  addressed  the  actual  choices  programmers  can  use  when  entering  a 
NAF  project.  The  survey  respondents  “agree”  with  all  three  parts  of  the  question  and  that 
the  new  choices  for  “Program  Type,”  “Fund  Type,”  and  “Type  of  Work”  would  benefit 
NAF  programming.  Since  no  additional  comments  were  provided,  it  can  be  assumed  that 
most  programmers  agree  that  no  changes  should  be  made  to  the  proposed  list. 

The  recommended  changes  shown  in  questions  G22  and  G23  dealt  with 
modifying  the  DD  Form  1391  to  improve  NAF  programming.  Base-level  programmers 
“agree”  that  these  changes  will  enhance  the  NAF  documentation  process.  Question  G22 
addressed  the  addition  of  two  line  items  related  to  NAF  programming  and  question  G23 
covered  change  to  the  order  of  the  line  items  in  the  DD  Form  1391  Detailed  Cost  Tab. 

4.3  Research  Question  2 

Recall  that  the  second  research  question  was:  “How  well  does  the  current  base 
level  ACES-PM  programming  process  support  the  new  ACES-PM  NAF  business  rule 
changes?”  Nine  survey  questions  were  used  to  answer  this  research  question.  These 
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questions  were  developed  to  forecast  whether  there  would  be  potential  problems  to  the 
proposed  changes  due  to  current  base-level  programming  procedures.  The  five  Likert 
Scale  questions  shown  in  Table  8  were  used  to  measure  how  often  certain  procedures  are 
done  at  base  level.  For  the  specific  content  of  each  question,  the  reader  is  referred  to 
Appendix  B. 


Table  8.  Descriptive  Statistics  for  Likert  Scale  Questions  (Research  Question  2) 


SURVEY 

QUESTION 

MEDIAN 

IRQ 

G8 

4 

2 

G9 

3 

1.5 

G10 

4 

2 

Gil 

6 

2 

G17 

4 

2.5 

Questions  G8  and  G9  involved  updating  the  project  manager  information  and 
roles  within  the  ACES-PM  “managers”  screen.  The  programmers  were  “undecided” 
about  Question  G8,  which  indicates  that  programmers  often  do  not  update  their  project 
manager  infonnation.  The  IQR  of  2  indicates  that  an  equal  number  of  programmers 
“agree”  and  “disagree”  with  the  question;  in  fact,  17  out  of  35  respondents  “disagree”  to 
some  extent  that  they  complete  the  manager  infonnation  for  new  projects.  For  question 
G9,  the  median  score  of  the  responses  was  3,  meaning  that  the  respondents  “disagree” 
with  the  statement  that  they  update  the  project  manager  information  when  they  release 
the  project  to  another  programmer.  This  result  demonstrates  there  is  a  potential  problem 
if  the  manager  information  is  used  to  send  the  automatic  email  notifications  discussed  in 
earlier  questions. 
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Question  G10  had  a  median  score  of  4,  thereby  showing  that  programmers  are 
“undecided”  about  using  the  milestone  tab  once  it  is  updated  to  better  fit  the  NAF 
programming  process.  The  IQR  of  2  also  indicates  the  amount  of  spread  in  the 
responses;  many  respondents  either  “strongly  agree”  or  “strongly  disagree”  with  the 
utilization  of  the  milestone  tab  after  it  is  modified. 

Question  Gil  had  median  score  of  6  and  an  IQR  of  2.  Most  programmers  believe 
it  is  important  to  detennine  if  a  NAF  project  has  a  companion  project  and  to  document  it. 
Only  2  programmers  felt  it  was  somewhat  unnecessary  to  document  whether  companion 
projects  exist  when  inputting  NAF  projects  into  ACES-PM. 

Question  G17  had  the  most  spread  in  its  answers  out  of  all  the  Likert  Scale 
questions.  It  is  also  one  of  the  more  controversial  issues  since  it  could  create  more  work 
for  project  programmers.  It  had  a  median  score  of  4  with  an  IRQ  of  2.5.  Thus,  many 
programmers  either  “strongly  agree”  or  “strongly  disagree”  that  they  should  be 
responsible  for  attaching  INVS  electronic  documents  to  the  ACES-PM  NAF  project  file. 

The  descriptive  statistics  for  the  multiple  choice  questions  supporting  research 
question  2  are  shown  in  Table  9.  The  results  from  question  Ml  help  solidify  the  idea  that 
many  programmers  are  not  satisfied  with  the  existing  DD  Form  1391  template.  For  one 
reason  or  another,  only  29  percent  of  the  respondents  solely  use  the  ACES-PM  template. 
Over  34  percent  admit  that  they  only  use  a  Microsoft  Word  template.  Question  M2 
examined  the  frequency  with  which  programmers  check  the  status  and  accuracy  of 
existing  NAF  projects.  The  results  indicate  that  only  14  percent  of  the  respondents  stated 
they  check  the  status  of  NAF  projects  more  than  once  a  month.  Question  M3  considered 
the  frequency  with  which  non-CE  users  wanted  to  see  the  status  of  installation  projects. 
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Almost  85  percent  of  the  respondents  stated  that  other  users  ask  to  see  projects  only  once 
a  month  or  longer.  Question  M4  yielded  some  interesting  results.  Seventeen  percent  of 
the  respondents  stated  that  they  would  not  use  the  ACES-PM  template  even  if  it  were 
updated  to  better  support  the  NAF  programming  process;  they  would  continue  to  use  the 
non  ACES-PM  option  currently  being  used  to  complete  the  DD  Form  1391. 


Table  9.  Descriptive  Statistics  for  Multiple  Choice  Questions  (Research  Question  2) 


Ml 

I  currently  li 

1  out  NAF  D 

D  Form  1391’s  for  NAF  projects  using: 

CHOICES 

ACES-PM 

Template 

MS  Word 
Template 

Both 

Other 

NUMBER 

10 

12 

11 

2 

FREQUENCY 

29% 

34% 

31% 

6% 

M2 

How  often  do  you  check  t 
installation’s  NAF  project 

re  status  an 
s  in  ACES- 

d  accuracy  of  your 

PM? 

CHOICES 

>  Once  a 
Week 

Once  a 
Week 

>  Once  a 
Month 

Once  a 
Month 

<  Once 
a  Month 

Never 

NUMBER 

0 

1 

4 

18 

9 

3 

FREQUENCY 

0% 

3% 

11% 

51% 

26% 

9% 

M3 

How  often  do  other  users  ask  you  about  the  status  and  accuracy  of 
your  installations  NAF  projects  in  ACES-PM? 

CHOICES 

>  Once  a 
Week 

Once  a 
Week 

>  Once  a 
Month 

Once  a 
Month 

<  Once 
a  Month 

Never 

NUMBER 

0 

2 

3 

19 

7 

4 

FREQUENCY 

0% 

6% 

9% 

54% 

20% 

11% 

M4 

If  the  current  template  is  revised  to  better  fit  the  NAF  process,  will 
you  use  the  new  template  to  input  the  initial  DD  Form  1391 
information? 

CHOICES 

Yes 

No 

NUMBER 

29 

6 

FREQUENCY 

83% 

17% 

4.4  Research  Question  3 

Recall  that  the  third  research  question  was,  “How  well  trained  are  current  base- 
level  programmers  in  NAF  programming  and  NAF  programming  in  ACES-PM?”  The 
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first  five  Likert  Scale  Questions,  G1  through  G5,  were  used  to  answer  this  research 
question.  The  main  purpose  of  this  research  question  was  to  detennine  the  current  status 
of  programming  knowledge  at  the  base  level.  The  question  examined  NAF  programming 
in  general  and  NAF  programming  within  ACES-PM  specifically.  Besides  measuring  the 
training  aspect,  the  five  questions  also  helped  establish  the  experience  and  knowledge  of 
the  data  sample.  Table  10  shows  the  median  scores  and  IQR  values  for  the  survey 
questions.  For  the  specific  content  of  each  question,  the  reader  is  referred  to  Appendix  B. 


Table  10.  Descriptive  Statistics  for  Likert  Scale  Questions  (Research  Question  3) 


SURVEY 

QUESTION 

MEDIAN 

IQR 

G1 

5 

2 

G2 

4 

2 

G3 

5 

1 

G4 

5 

1 

G5 

5 

0.5 

All  of  the  questions  except  G2  had  a  median  score  of  5,  indicating  that  the 
respondents  “agree”  with  the  statement.  Question  G2  had  a  median  score  of  4,  meaning 
that  programmers  were  “undecided”  about  whether  they  had  formal  training  in 
programming  NAF  projects  in  ACES-PM.  However,  question  G2  also  had  a  large  IQR. 
This  indicates  that  there  were  a  number  of  programmers  who  “completely  disagree”  that 
they  had  adequate  formal  training  with  respect  to  NAF  project  programming  in  ACES- 
PM.  Question  G1  showed  that  programmers  “agree”  that  they  have  had  adequate  fonnal 
training  in  NAF  project  programming.  The  results  of  questions  G3  and  G4  showed  that 
regardless  of  formal  training,  programmers  know  how  to  properly  program  NAF  projects 
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in  ACES-PM.  Question  G5  has  an  IQR  of  only  0.5,  which  means  that  the  majority  of 
programmers  agree  that  they  have  the  resources  available  to  answer  their  ACES-PM 
programming  questions. 

4.5  Chapter  Summary 

The  proposed  NAF  programming  fields  seem  to  be  accepted  by  the  majority  of 
ACES-PM  users.  The  results  of  a  few  of  the  questions  need  to  be  investigated  further  to 
determine  if  further  action  is  needed.  Some  of  the  answers  received  from  several  of  the 
questions  regarding  current  base-level  programming  procedures  could  present  some 
problems  when  the  changes  are  implemented.  These  issues  will  be  addressed  in  Chapter 
5,  where  a  summary  of  the  findings  is  presented  along  with  recommendations  for  the 
ACES-PM  IPT. 
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5.  CONCLUSIONS  AND  RECOMMENDATIONS 


This  chapter  provides  a  summary  of  the  research  findings  and  conclusions  to  each 
of  the  research  questions.  The  limitations  of  the  research  are  addressed  along  with 
recommendations  for  further  research.  Finally,  a  closing  summary  is  presented  about  the 
research  in  general. 

5.1  Research  Findings  and  Conclusions 

5.1.1  Research  Question  1 

Research  question  1  investigated  whether  specific  Automated  Civil  Engineer 
System  Project  Management  Module  (ACES-PM)  non-appropriated  funds  (NAF) 
business  rule  changes,  proposed  by  the  integrated  process  team  (IPT),  would  be 
beneficial  to  NAF  programming.  The  list  of  the  IPT’s  proposed  changes  is  shown  in 
Appendix  A.  Note  again  that  only  the  proposed  changes  labeled  with  an  asterisk  (*)  are 
addressed  in  the  survey  questions  pertaining  to  research  question  1.  Overall,  13  out  of  14 
changes  addressed  by  the  survey  questions  are  “agreed”  or  “strongly  agreed”  to  be 
beneficial  to  NAF  programming  in  ACES-PM.  These  results  are  promising  since  it 
shows  the  IPT  concentrated  only  on  those  business  rules  that  are  the  most  important  in 
tenns  of  improving  NAF  programming.  However,  a  thorough  analysis  of  each  survey 
question  coupled  with  some  of  the  results  from  research  question  2  yielded  some 
interesting  issues  that  should  be  further  looked  into  by  the  IPT. 

There  is  a  concern  that  NAF  projects  get  held  up  during  the  funding  process  due 
to  confusion  and  inefficiency  that  is  created  as  project  user  rights  switch  numerous  times 
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between  responsible  organizations  (6).  To  help  address  this  problem,  the  IPT  suggested 
the  use  of  automated  emails  to  alert  program  managers  of  the  existing  project  status  and 
if  an  action  is  required  to  further  the  funding  process.  Programmers  “agree”  that  email 
notifications  would  be  beneficial  to  the  NAF  programming  process.  The  results  of  the 
survey  also  revealed  that  most  programmers  view  the  status  of  NAF  projects  at  an 
average  less  than  once  a  month.  Since  the  frequency  in  which  NAF  projects  are  checked 
is  so  low,  an  email  notification  is  a  great  idea.  Besides  a  resourceful  method  to  provide 
instant  project  updates  to  project  managers,  an  automatic  email  system  could  notify  a 
given  agency  that  an  action  is  required  on  their  part  to  further  the  funding  process.  This 
notification  becomes  even  more  crucial  as  deadlines  draw  near  and  the  project  is  still  not 
approved  for  current  fiscal  year  funding. 

The  use  of  emails  might  not  be  the  best  way  to  notify  programmers  when  the 
status  of  a  project  has  changed.  The  success  of  this  concept  depends  on  whether  the 
project  mangers  update  their  personal  infonnation  in  the  manager’s  role  tab.  Most 
programmers  are  “undecided”  as  to  whether  they  input  the  required  PM  information 
when  taking  over  a  new  project,  and  “disagree”  that  they  help  ensure  the  new  PM 
information  is  updated  when  they  leave  a  project.  These  results  strongly  indicate  that 
there  is  potential  for  incorrect  information,  more  specifically  email  addresses,  to  exist  in 
the  PM  tab.  If  the  correct  email  is  not  inputted  at  the  beginning  of  the  project,  or  the 
project  manager  changes  and  the  email  address  is  not  updated,  the  notification  will  be 
lost.  The  current  ACES  system  can  only  send  a  notification  email  once.  If  the 
recipient’s  address  is  not  current  and  valid,  the  email  is  lost.  However,  at  best,  most 
programmers  are  only  checking  projects  on  a  monthly  basis. 
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The  best  way  to  ensure  that  project  approval  does  not  get  held  up  is  to  use  a 
combination  of  both  notification  methods.  The  idea  to  implement  a  history  scroll  down 
data  field  that  shows  all  project  status  changes,  completion  dates,  and  who  accomplished 
the  change  will  allow  programmers  to  easily  catch  up  on  the  status  of  a  project.  This  tool 
would  also  be  very  helpful  for  project  managers  who  take  over  a  project  in  the  middle  of 
the  funding  process.  For  those  programmers  who  are  managing  numerous  projects,  an 
email  can  also  be  sent  out  to  help  ensure  that  the  project  does  not  get  stalled  in  one  of  the 
approval  phases.  Email  notification  should  only  be  used  to  tell  a  project  manager  that  an 
action  is  required  on  his  or  her  part.  Instruction  and  guidance  must  be  incorporated  into 
the  web-based  training  that  stresses  the  importance  of  updating  the  PM  tab  information 
for  each  project.  Supervisors  must  also  be  involved  in  the  updating  process  since  there  is 
a  great  potential  to  lose  information  when  the  PM  is  changed.  Finally,  for  last  resort, 
project  managers  at  all  approval  levels  can  help  facilitate  the  funding  process  by 
forwarding  emails  and  making  telephone  calls. 

ACES-PM  has  the  ability  to  attach  different  types  of  electronic  files  to  the  project 
file  (6).  The  use  of  paperless  methods  to  report  all  NAF  project  information  is  a  future 
goal  that  is  only  obtainable  one  step  at  a  time  (6).  Attaching  the  INVS  to  the  NAF 
project  folder  is  a  great  way  to  start  taking  advantage  of  the  automated  information 
system  (AIS)  features  available  to  CE  programmers.  Since  most  programmers  “agree” 
that  the  INVS  should  be  attached  electronically,  the  responsibility  of  attaching  it  needs  to 
be  specified  at  the  Major  Command  (MAJCOM)  level  and  consistently  delegated  down 
to  the  base-level.  Even  though  programmers  feel  that  it  would  be  beneficial  to  attach  the 
INVS,  they  are  “undecided”  as  to  whether  programmers  should  be  required  to  accomplish 
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the  task  within  ACES-PM.  If  the  ACES-PM  IPT  truly  wants  this  option  to  be  utilized  by 
the  programming  community,  new  guidelines  and  training  need  to  be  implemented. 

Since  the  Services  community  responsible  for  the  IN  VS,  they  should  be  delegated  the  job 
of  scanning  in  the  documents,  and  programmers  should  be  required  to  attach  them.  If  this 
relationship  can  be  accepted  and  executed  properly  with  the  INVS,  consideration  should 
be  brought  up  in  regards  to  also  having  other  documents  like  the  Commander’s 
Certificate  (CC)  and  Needs  Assessment  Study  (NAS)  attached  to  the  project  folder.  The 
procedure  to  attach  documents  to  the  project  folder  must  be  covered  in  the  web-based 
training  curriculum. 

Programmers  believe  it  would  be  beneficial  to  allow  Services  to  have  read-rights 
to  all  NAF  projects.  This  is  obvious  since  programmers  have  more  work  to  accomplish 
when  Services  personnel  call  and  ask  for  updates.  However,  Services  only  inquire  about 
projects  on  a  month-to-month  basis.  It  would  not  be  worth  the  cost  and  time  to  fully 
equip  Services  with  access  to  ACES.  Therefore,  it  would  be  less  expensive  to  keep  the 
current  system  where  Services  calls  or  emails  the  programming  office  once  a  month  to 
ask  for  updates.  It  is  part  of  the  programming  office’s  responsibility  to  give  project 
updates  to  other  base  organizations  on  a  need-to-know  basis.  This  interaction  will  also 
keep  the  using  organizations  on  the  same  page  by  promoting  better  communication  and 
continuity  with  project  planning. 

Most  programmers  do  not  feel  it  is  necessary  for  MAJCOMs  to  release  project 
rights  back  to  base  level  when  projects  are  not  selected  for  the  next  step  in  the  approval 
process.  Thus,  most  MAJCOMs  should  hold  onto  the  user  rights  to  the  project  files. 

This  will  allow  MAJCOMs  to  more  effectively  manage  their  multitude  of  NAF  projects, 
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and  cut  down  on  the  amount  of  project  transferring  that  is  required.  However, 
MAJCOMs  will  eventually  have  to  give  installations  rights  to  unfunded  projects  to  ensure 
that  they  are  updated  for  the  project  call  the  following  year. 

It  is  crucial  that  there  is  no  confusion  in  how  a  NAF  project  is  determined  to  have 
a  “program  type”  of  NAF.  All  the  business  rules  associated  with  NAF  programming  will 
be  initiated  once  the  programmer  specifies  on  the  project  add  screen  that  the  “program 
type”  is  NAF.  Base-level  programmers  “agreed”  that  determining  the  “program  type” 
along  with  the  “fund  type”  and  “type  of  work”  are  a  lot  easier  with  the  new  categories 
proposed  by  the  IPT.  Having  specific,  concise  selections  for  these  three  groups  of  data 
will  help  ensure  that  the  projects  are  inputted  correctly  from  the  beginning.  The  integrity 
of  the  NAF  project  data  will  subsequently  improve  making  queries  for  certain  types  of 
NAF  projects  more  efficient  and  accurate  for  all  using  agencies  involved. 

5,1.2  Research  Question  2 

Research  question  2  examined  whether  the  current  programming  processes 
exhibited  by  base-level  programmers  would  partially  impede  the  successful 
implementation  of  the  proposed  business  rule  changes.  The  results  to  this  research 
question  produced  some  interesting  observations  and  brought  up  many  potential  problems 
areas  that  will  need  to  be  dealt  with  by  the  IPT.  In  general,  most  programmers  do  not  pay 
enough  attention  to  the  status  of  their  respective  projects,  and  do  not  constantly  update 
information  in  the  PM  screen  and  the  Milestone  tab.  The  results  also  show  signs  of  end 
user  resistance  to  using  ACES  since  17  percent  of  programmers  stated  they  will  continue 
to  use  non-ACES-PM  templates  to  fill  out  DD  Form  1391  information.  Many  of  the 
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potential  problems  associated  with  current  programming  procedures  were  addressed  in 
conjunction  with  the  proposed  changes  shown  in  the  previous  section.  However,  the 
results  obtained  in  this  research  question  presented  some  additional  concerns  and 
recommendations  for  the  IPT. 

The  consistent  use  of  the  ACES-PM  Milestone  Tab  would  dramatically  help  in 
the  effective  management  of  NAF  projects.  However,  programmers  are  “undecided”  as 
to  whether  they  would  even  use  the  tab  even  if  it  was  modified  to  better  fit  the  NAF 
programming  process.  The  Air  Force  Civil  Engineer  Support  Agency  (AFCESA)  is 
determining  what  types  of  milestones  should  be  added  and  deleted  to  create  a  NAF 
Milestone  Tab.  If  more  consistent  use  of  the  tab  is  required  for  future  programming 
requirements,  AFCESA  should  ensure  that  is  it  updated  to  the  best  extent  possible.  The 
Milestone  Tab  can  also  be  an  effective  way  to  view  the  status  of  projects  as  long  as  it  is 
consistently  updated  by  the  PM. 

The  ability  to  identify  and  program  appropriated  funds  (APF)  companion  projects 
while  inputting  a  NAF  project’s  initial  requirements  would  be  very  beneficial  to  the 
programming  process.  Programmers  put  forth  a  thorough  effort  in  trying  to  determine  if 
an  APF  companion  project  exists.  Thus,  creating  a  section  that  allows  programmers  to 
state  whether  an  APF  companion  project  exists,  along  with  a  project  number  and 
programmed  amount  would  greatly  help  eliminate  problems  in  the  NAF  funding  process. 
A  mandatory  check  box  will  force  the  programmers  to  determine  whether  an  APF 
companion  project  needs  to  be  programmed  in  conjunction  with  the  NAF  project.  If  the 
ability  to  link  the  related  project  files  does  not  exist,  the  addition  of  APF  companion 
project  number  and  PA  data  fields  would  greatly  assist  in  the  funding  process. 
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Programmers  show  a  great  effort  in  determining  APF  companion  projects;  the  more 
information  they  can  provide  during  initial  project  input,  the  less  of  a  chance  the  project 
will  be  held  up  during  later  approval  levels. 

Guidance  and  instruction  are  required  from  the  Air  Staff  level  to  help  enforce  that 
all  base-level  programmers  use  the  ACES-PM  template  to  input  DD  Fonn  1391  NAF 
project  infonnation,  especially  once  it  is  revised  to  better  reflect  NAF  programming 
requirements.  The  results  of  the  survey  suggested  that  programmers  would  continue  to 
use  other  templates  for  completing  DD  Form  1391s.  This  presents  a  problem  with  data 
compatibility  and  integrity  issues,  along  with  work  inefficiency,  as  the  infonnation  will 
need  to  be  re-created  in  ACES-PM  before  the  project  can  be  sent  to  the  MAJCOM. 
Besides  making  the  template  better  minor  the  NAF  programming  process,  the  Standard 
Systems  Group  (SSG)  has  to  investigate  the  other  potential  reasons  that  are  making 
programmers  hesitate  to  use  the  ACES-PM  template.  It  should  be  a  mandatory 
requirement  that  all  CE  programmers,  from  base  level  to  Air  Staff,  solely  use  the  ACES- 
PM  DD  Form  1391  template.  Air  Staff  and  MAJCOMs  have  to  develop  guidelines  and 
enforce  them  throughout  the  programming  community.  Once  these  actions  are  taken,  the 
problem  of  trying  to  get  different  DD  Form  1391s  to  match  up,  or  wondering  what  the 
most  current  and  correct  form  is,  will  be  eliminated.  The  IPT’s  goal  should  be  to 
eventually  have  all  DD  Fonn  1391s  produced  from  one  system  where  user  rights  will 
guarantee  consistency  and  accuracy  of  the  data. 
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5.1.3  Research  Question  3 

Research  Question  3  investigated  the  training  aspects  of  ACES-PM  and  NAF 
programming.  The  results  of  the  survey  showed  that  most  programmers  are  confident  in 
their  NAF  programming  knowledge  and  ability  to  input  information  into  ACES-PM. 
However,  they  were  “undecided”  as  to  whether  they  have  had  formal  training  on  NAF 
programming  in  ACES-PM.  Nonetheless,  programmers  “agree”  that  they  properly  know 
how  to  manage  NAF  projects  in  ACES-PM.  AFCESA  and  the  SSG  are  currently 
implementing  a  new  web-based  program  that  will  become  the  formal  training  for  ACES- 
PM.  Hopefully,  the  ease  and  accessibility  of  the  training  program  will  allow  all 
programmers  to  feel  they  have  had  the  proper  training  with  ACES-PM  and  NAF  projects. 

When  the  web-based  program  becomes  operational  it  is  essential  that  it  be 
advertised  and  promoted  within  the  programming  community.  Endorsement  by 
MAJCOM  programmers  will  help  increase  the  amount  of  programmers  who  use  the 
training  system.  Even  though  most  programmers  “agree”  there  are  adequate  resources 
available  to  get  questions  answered,  having  a  web-based  learning  system  will 
significantly  improve  the  ACES  training  issue.  Additionally,  when  changes  are  made 
within  the  ACES  module,  the  training  site  should  be  updated  to  reflect  the  changes.  For 
instance,  the  changes  to  the  NAF  programming  business  rules  need  to  be  reflected  in  the 
first  version  of  the  ACES-PM  training  site.  Once  the  proposed  changes  are  implemented, 
it  will  be  important  that  the  ACES-IPT  incorporate  the  new  NAF  procedures  into  the 
web-based  training  program.  If  the  training  program  becomes  outdated,  it  will  not  be 
utilized  to  its  full  potential.  Also,  since  NAF  programming  business  rules  are  extremely 
different  from  other  project  types,  NAF  specific  training  should  created  within  the  web- 
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based  training  program.  A  stand-alone  NAF  training  section  will  allow  programmers  to 
become  current  on  the  all  business  rule  changes  more  efficiently. 

5.2  Limitations  of  the  Research 

This  research  has  several  areas  of  limitations.  The  most  influential  is  the  lack  of 
data.  The  final  data  set  only  contained  35  responses  from  base-level  programmers.  This 
number  is  well  below  the  recommended  sample  size  of  168,  which  had  an  optimistic  50 
percent  response  rate.  The  lack  of  data  brings  more  error  and  uncertainty  into  the  data 
analysis.  Additionally,  the  low  sample  size  may  have  contributed  to  not  receiving  any 
comments  from  the  survey  questions  and  the  inability  to  perform  content  analysis. 

Despite  the  statistical  limitations,  from  an  operational  standpoint,  the  35  responses 
represented  more  feedback  on  potential  ACES-PM  business  rule  changes  than  ever 
received  before  by  the  IPT  (43).  Since  the  inception  of  ACES-PM,  the  IPT  has  had  a 
significant  problem  trying  to  obtain  feedback  from  the  programming  community  on  their 
proposed  business  rule  changes. 

There  are  three  reasons  for  the  low  response  rate.  First,  the  survey  was  sent  out  in 
early  December,  allowing  only  one  and  a  half  months  of  data  collection.  Second,  with 
the  timing  of  the  survey  administration  being  right  before  the  holidays,  many  potential 
respondents  could  have  been  out  of  the  office.  Finally,  the  process  used  to  distribute  the 
survey  was  not  efficient.  It  was  assumed  that  MAJCOMs  would  quickly  forward  the 
survey  link  to  their  base-level  program  offices  and  stress  the  importance  of  the  survey  to 
the  CE  programming  community.  Each  MAJCOM  had  the  directions  to  reply  to  the 
email  stating  that  they  did  indeed  forward  the  link  to  their  bases,  yet  only  two  of  the  eight 
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accomplished  the  task.  The  lack  of  email  responses  raised  the  question  as  to  whether  or 
not  the  survey  link  was  even  forwarded  to  some  of  the  installations. 

The  method  in  which  the  data  was  collected  also  presents  a  limitation.  The 
approximate  number  of  programmers  who  had  recent  NAF  programming  experience  in 
ACES-PM  could  not  be  calculated  and  contact  information  for  each  could  not  be  found 
easily.  The  method  of  sending  the  survey  link  to  ah  MAJCOMs  made  a  lot  of  sense,  but 
it  created  a  sample  that  was  not  random.  The  unbalanced  weighting  in  terms  of 
MAJCOMs  shows  this  limitation.  When  responses  were  not  being  achieved  using  the 
original  method  of  distribution,  other  methods  were  created.  These  methods  also  led  to 
biased  responses  since  the  email  was  directly  emailed  to  ah  known  CE  programmers, 
instead  of  randomly  selected.  If  this  type  of  research  is  to  be  accomplished  again,  a  well 
thought  out  process  for  getting  the  survey  to  a  sufficient  amount  of  programmers  in  a 
random  manner  should  be  determined  at  the  start  of  the  research.  The  ACES-PM  IPT, 
AFCESA,  or  the  SSG  might  have  a  list  of  ah  active  ACES-PM  users,  which  could  be 
used  to  ensure  a  random  result  is  achieved. 

The  construction  of  the  survey  was  also  a  limitation.  Since  it  was  created 
specifically  for  this  research  effort,  reliability  of  the  instrument  was  not  previously 
known.  The  manner  in  which  the  questions  were  developed,  coupled  with  a  time 
constraint,  also  inhibited  the  ability  to  measure  the  external  reliability  of  the  instrument. 
The  questions  also  could  have  been  shorter  and  more  concise  in  order  to  promote  the 
respondent  to  give  comments  on  the  survey  questions. 

The  last  limitation  involves  the  actual  research  instrument.  Even  though  a  survey 
guarantees  more  data  is  collected,  performing  a  few  in-depth  case  studies  could  provide 
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other  forms  of  information.  This  concept  might  explain  the  lack  of  comments  with  each 
of  the  survey  questions.  The  lack  of  time  to  get  the  survey  distributed  also  decreased  the 
amount  of  time  that  was  spent  testing  it  with  a  pilot  study.  Furthermore,  the  literature 
could  have  been  reviewed  in  more  detail  to  detennine  if  there  are  proven  fonnats  for 
writing  survey  questions  that  deal  with  improving  existing  AIS. 

5.3  Areas  of  Further  Research 

The  improvement  of  AISs  is  a  continual  process  that  goes  on  for  the  whole 
duration  of  a  system’s  life  cycle  (41 : 12).  No  AIS  is  perfect  since  the  needs  of  the 
organization  are  always  changing.  Improving  NAF  programming  in  ACES-PM  can  be 
re-investigated  when  the  requirements  of  the  system’s  end  users  change  or  when  new 
ideas  regarding  the  programming  of  NAF  projects  are  developed. 

From  the  results  of  this  research,  it  might  be  useful  to  survey  all  ACES-PM  users 
to  determine  reasons  for  end  user  resistance.  Some  programmers  are  hesitant  to  use 
ACES-PM  for  all  programming  tasks  and  do  not  have  confidence  the  system  will 
increase  work  efficiency.  The  six  respondents  who  stated  they  would  continue  to  use 
other  non- ACES-PM  templates  reflects  the  current  situation  that  some  programmers  do 
not  want  to  change  their  way  of  doing  business.  Potential  reasons  could  be  collected 
from  an  unbiased  source  to  produce  useful  results  for  the  ACES-PM  IPT  to  research 
further. 

With  the  limitations  in  mind,  another  way  to  explore  improvement  to  ACES-PM 
and  other  modules  could  include  a  multi-case  study  approach  that  covers  one  base-level 
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programming  office  for  each  MAJCOM.  This  method  might  allow  for  more  specific 
issues  to  be  addressed  and  enable  the  respondents  to  talk  about  their  specific  concerns. 

All  ACES  modules,  no  matter  what  method  is  used,  should  be  investigated  from  an 
unbiased  viewpoint.  All  eight  modules  will  have  problems,  and  many  end  users  have  the 
knowledge  and  experience  to  correct  the  situation,  but  think  they  are  not  in  the  position  to 
make  a  difference. 

This  research  has  provided  further  recommendations  on  improving  NAF 
programming  in  ACES-PM.  However,  there  are  many  other  areas  of  ACES-PM  that  can 
be  investigated  at  the  base-level  to  find  ways  to  further  improve  the  module.  For 
example,  the  operations  and  maintenance  (O&M)  and  Military  Construction  (MILCON) 
project  business  rules  and  their  related  project  tabs  can  be  analyzed  for  areas  of  further 
improvement  using  the  same  type  of  survey  method.  Questionnaires  can  be  created  based 
on  the  current  business  rule  problems  and  administered  to  all  base-level  programmers. 
This  process  would  give  the  IPT  more  recommendations  on  how  to  improve 
programming  for  all  types  of  projects  in  ACES-PM. 

5.4  Final  Summary 

It  is  important  that  the  organizations  responsible  for  the  ultimate  success  of 
ACES,  to  include  NAF  programming  within  ACES-PM,  continue  to  improve  the  current 
AIS  structure  and  business  rules.  In  order  to  ensure  NAF  programming  in  ACES-PM 
remains  effective  and  efficient,  the  requirements  and  recommendations  of  the  AIS  end 
user  will  have  to  continually  be  addressed  and  supported  by  the  IPT.  This  research 
provided  a  new  approach  to  evaluating  AIS  changes  once  implemented  within  an 
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organization.  In  conclusion,  this  research  attempted  to  improve  NAF  programming 
within  ACES  and  created  another  method  to  how  business  rule  problems  can  be  solved 
via  end  user  input. 
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ACRONYMS  &  ABBREVIATIONS 


AAFES-  Army  and  Air  Force  Exchange  Service 
ACES-  Automated  Civil  Engineering  System 

ACES-EM-  Automated  Civil  Engineer  System  Environmental  Management  Module 
ACES-EO-  Automated  Civil  Engineer  System  Explosive  Ordinance  Module 
ACES-FD-  Automated  Civil  Engineer  System  Fire  Department  Module 
ACES-HM-  Automated  Civil  Engineer  System  Housing  Module 
ACES-OPS-  Automated  Civil  Engineer  System  Operations  Module 
ACES-PM  -  Automated  Civil  Engineering  System  Project  Management  Module 
ACES-PR-  Automated  Civil  Engineer  System  Personnel  Readiness  Module 
ACES-RP-  Automated  Civil  Engineer  System  Real  Property  Module 
ADAL-  Add/Alter 
AE-  Architect-Engineer 

AFCESA-  Air  Force  Civil  Engineer  Support  Agency 
AFCIC-  Air  Force  C4  Infonnation  Command 
AFI-  Air  Force  Instruction 

AF/ILEC-  Office  of  the  Civil  Engineer,  Engineering  Division 

AFIT-  Air  Force  Institute  of  Technology 

AFSVA-  Air  Force  Services  Agency 

AIS-  Automated  Information  System 

APF-  Appropriated  Funds 

ASG-  Automated  Steering  Group 

BEAMS-  Base  Engineer  Automated  System 
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C4-  Command,  Control,  Communications,  and  Computer 
CC-  Commander’s  Certificate 
CCB-  Configuration  Control  Board 
CE-  Civil  Engineering 

CRSD-  C4  Systems  Requirements  Document 
D3  -  Deficiency  Detail  Data 
DECA-  Defense  Commissary  Agency 
DECC-  Defense  Enterprise  Computing  System 
DI-  Design  Instruction 

DMRD-  Defense  Management  Review  Decision 

DoD-  Department  of  Defense 

DOS-  Disk  Operating  System 

FFE-  Furnishings,  Fixtures,  and  Equipment 

FIM-  Facility  Investment  Metric 

GS-  General  Schedule 

HQ-  Headquarters 

IN  VS-  Internal  Needs  Validation  Study 
IPT-  Integrated  Process  Team 
IQR-  Inter  Quartile  Range 
IS-  Infonnation  System 

IWIMS-  Interim  Work  Information  Management  System 
MAJCOM-  Major  Command 
MILCON-  Military  Construction 
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MWRS-  Morale,  Welfare,  Recreation,  and  Services 

NAF-  Nonappropriated  Fund 

NAFI-  Nonappropriated  Fund  Instrumentality 

NAS-  Needs  Assessment  Study 

O&M-  Operations  and  Maintenance 

OSD-  Office  of  the  Secretary  of  Defense 

PA-  Program  Amount 

SAF-  Secretary  of  the  Air  Force 

SON-  Survey  Control  Number 

SDLC  -  System  Development  Life  Cycle 

SEP  -  Systems  Engineering  Process 

SIOH-  Supervision,  Inspection,  and  Overhead 

SSG-  Standard  Systems  Group 

TID-  Technology  Integration  Division 

TTC-  Tactical  Training  Center 

WIMS-  Work  Information  Management  System 
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APPENDIX  A.  PROPOSED  ACES-PM  NAF  BUSINESS  RULE  CHANGES 
(*  means  proposed  change  addressed  in  research  question  1) 


Item  # 

ACES-PM  Location 

Description  of  Proposed  Change 

1 

Programming  Tab 

Approval  authority  (Local,  MAJCOM,  AF)  shown 
by  checking  one  of  the  three  respective  boxes 

*2 

Programming  Tab 

A  “Pre-INVS”  status  box  will  be  added  to  show 
projects  whose  importance  does  not  warrant  the 
start  of  the  IN  VS.  This  box  will  also  state  when 
the  INVS  was  initiated  and  completed. 

*3 

Program  Pull  Down 

Menu 

The  INVS  document  will  be  electronically  attached 
to  the  project  file  with  digital  signatures  for  FM, 

CE,  and  SV  Commanders. 

*4 

Programming  Tab 

A  status  field  will  be  added  so  programmers  can 
quickly  see  the  status  of  a  project  to  include  who 
has  user  rights  and  what  is  the  next  required  action 
in  the  funding  process 

*5 

Programming  Tab 

Project  selection  for  the  next  step  in  the  funding 
process,  i.e.,  being  selected  for  the  INVS,  NAS,  or 
design  will  be  accomplished  by  adding  “not 
selected/selected”  boxes  to  be  check  by  MAJCOM 
and  AFSVA.  If  the  project  is  a  “non-select”,  for 
the  NAS,  it  automatically  transfers  back  to  the 
MAJCOM  and  to  base  if  desired. 

*6 

Project  Add  Screen 

Add  a  field  that  show  if  there  is  a  companion 
project,  and  if  so,  the  project  #  and  PA  can  be  filled 
in.  The  programmer  cannot  create  the  project 
without  checking  “yes”  or  “no”. 

7 

Design  Tab 

The  four  DI’s  will  be  listed  along  with  their 
approval  dates.  Once  the  Office  of  the  Civil 
Engineer,  Engineering  Division  (AF/ILEC)  checks 
a  DI  off,  the  user  rights  transfer  back  to  the 
MAJCOM 

8 

Design  Tab 

When  the  method  of  design  is  Design-Bid-Build, 
the  default  design  %  is  35%.  When  it  Design- 
Build,  it  only  15%  when  AFSVA  racks  and  stacks 
the  projects. 

9 

Design  Tab 

When  the  project  is  at  %35  design  and  selected  for 
100%  design,  a  comments  page  and  an 
“approve/not  approve”  screen  should  be  added  to 
allow  AFSVA  and  MAJCOM  to  agree  on  the  final 
official  DD  Form  1391.  This  will  make  final 
approval  and  signatures  from  MAJCOM  more 
efficient.  A  validation  screen  will  be  added  to 
allow  MAJCOM  and  AFSVA  to  easily  discuss 
issues  with  the  final  1391. 
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*10 

General 

Email  notifications  will  be  utilized  to  give  users 
notification  when  a  project  requires  an  action  on 
their  part.  The  Milestone  Screen  can  automatically 
send  these  once  a  milestone  is  complete.  Email 
addresses  will  have  to  be  updated  on  the  Managers 
Screen  for  this  option  to  work. 

*11 

Programming  Tab 

Add  a  “current  status”  field  that  allows  users  to 
easily  scroll  through  the  history  of  events  that  have 
happened  with  a  project.  Information  will  include 
the  dates  the  event  took  place,  and  who  was  the 
user  responsible  for  the  event. 

12 

Funding  Tab 

Add  a  “funds  issued”  field  along  with  the  “date” 
field  that  is  filled  in  by  AFSVA  so  MAJCOM  and 
base  know  project  is  just  awaiting  bid  selection 

*13 

Programming  Tab 

Add  a  check  box  asking  if  the  facilities  board 
approved  the  project. 

*14 

Project  Add  Screen 

Change  “Program  Type  to  “NAF” 

*15 

Project  Add  Screen 

Change  “Fund  Type”  to  one  of  the  following 
choices:  AAFES,  DECA,  MWR,  Private,  Lodging, 
Multi.  Inserting  “NAF’  as  the  “Program  Type”  sets 
this  business  rule. 

*16 

Project  Add  Screen 

Change  “Type  of  Work”  to  one  of  the  following 
choices:  ADAL  (add/alter),  Construction, 
Maintenance/Repair,  Minor  Construction,  0  &  M. 
Inserting  “NAF’  as  the  “Program  Type”  sets  this 
business  rule. 

17 

Programming  Tab 

Grey  out  (make  non-active  field)  the  “PE”, 
“CE/PBD”  and  “Group”  data  fields. 

18 

Programming  Tab 

Change  to  “Environmental”  and  “MILCON”  check 
blocks  to  “Local”,  MAJCOM/AF”,  and  “AF”. 

19 

Programming  Tab 

Add  “Design”  and  “FFE”  as  subcategories  of  the 
“Unfunded  Amount”  field 

20 

Programming  Tab 

Change  “Excluded  Amount”  data  field  to  “APF 
Projects” 

21 

Programming  Tab 

Change  “Total  Amount”  data  field  to  “Total  NAF 
Investment”. 

22 

Project  Managers  Screen 

Add  AFSVA  project  Manager  as  one  of  the 
primary  roles. 

23 

Supplemental  Tab 

Add  AFSVA  as  part  of  the  “Base/MAJCOM/  and 

Air  Staff’  program  checklist.  When  these  boxes 
are  checked,  the  user  rights  transfer  to  the  next 
organization  in  the  funding  process.  Include  a  date 
box. 

24 

Supplemental  Tab 

Add  “AFSVA”  to  the  “PM”  field  to  assist  in 
transferring  user  rights. 
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*25 

Design  Tab 

Add  a  “Total  NAF  Investment”  field  which 
includes  the  FFE  and  design  cost.  “PA”  covers  the 
construction/contingency  and  Supervision, 
Inspection,  and  Overhead  (SIOH)  cost.  This 
should  also  be  shown  on  the  DD  Form  1391 

26 

Design  Tab 

Add  dates  for  when  each  DI  was  issues  by 

AF/ILEC. 

27 

Construction  Tab 

Change  business  rules  so  that  when  a  mod  is 
entered,  which  will  break  the  125%  cost  or  1 10% 
scope  threshold,  ACES  will  not  allow  the  mod  to 
go  through  until  the  project  is  sent  to  AF/ILEC. 

28 

Construction  Tab 

Add  an  alert  notification  when  a  project  hits  the  10 
and  20  %  over  cost  figure. 

*29 

Construction  Tab 

Add  business  rule  that  asks  the  MAJCOM  if  they 
want  to  transfer  user  rights  back  to  base  that  allows 
them  to  update  construction  complete  figures  and 
input  modifications. 

30 

Funding  Tab 

Under  “Fund  Indicator”  Tab,  change  choices  to: 
Local,  MAJCOM,  AF,  and  Mulit 

31 

Funding  Tab 

Add  “Total  NAF  Investment  Field” 

*32 

Detail  Cost  Sheet  (used 
to  fill  out  Block  9  of  DD 
Form  1391.)  These  data 
fields  will  be  put  into 
the  Funding  Tab. 

Change  the  sheet  to  the  following  setup: 

Contract  Amount  at  Award 
+  Modifications 
=  Current  Contract  Amount 
+  Contingency  Amount 
=  Subtotal 

+  SIOH  (%  of  subtotal) 

=  Subtotal  (PA  amount  used  for  thresholds) 

+  NAF  design  cost 
+  FFE  costs 
=  Total  NAF  investment 

Unfunded  Amount  (APF  non-add) 

Last  line  Blank 

33 

DD  Form  1391 

In  block  1 1  add  an  “additional”  slot  to  list 
companion  projects,  their  titles,  cost,  etc. 

*34 

Milestone  Tab 

AFSVA  will  create  a  whole  new  milestone  screen 
for  NAF  projects.  New  entries  will  include  INVS 
complete,  NAS  complete,  selected  for  35%  design, 
35%  design  complete,  DI’s  issues,  100%  design, 
etc. 

35 

General 

Base  level  services  should  be  granted  read-only 
rights  to  view  NAF  projects 
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APPENDIX  B.  SURVEY  QUESTIONS 


EVALUATION  OF  CURRENT  AUTOMATED  CIVIL  ENGINEERING  SYSTEM 

(ACES) 

NON-APPROPRIATED  FUNDS  (NAF)  PROJECT  PROGRAMMING  PROCEDURES 

USAF  SCN  03-124 


Purpose:  The  purpose  of  this  study  is  to  determine  whether  the  proposed  changes  to  the  current  NAF 
ACES-PM  programming  procedures  and  business  rides,  recently  proposed  by  the  ACES-PM  IPT,  are 
efficient  and  complete.  The  results  of  this  research  effort  will  be  used  to  give  the  ACES-PM  IPT 
recommendations  for  both  additions  and  deletions  to  the  proposed  changes,  along  with  an  idea  of  how  well 
these  changes  will  be  accepted  within  the  Air  Force  programming  community.  The  feedback  you  provide 
in  this  questionnaire  will  directly  aid  in  the  improvement  of  NAF  project  programming  throughout  the 
USAF. 

Participation:  We  would  greatly  appreciate  your  completing  this  survey.  Your  participation  is 
COMPLETELY  VOLUNTARY.  You  may  withdraw  from  this  study  at  any  time  without  penalty,  and  any 
data  that  has  been  collected  from  you,  as  long  as  that  data  is  identifiable,  can  be  withdrawn  by  contacting 
Captain  Joshua  Ligday.  Your  decision  to  participate  or  withdraw  will  not  jeopardize  your  relationship  with 
your  organization,  the  Air  Force  Institute  of  Technology,  the  Air  Force,  or  the  Department  of  Defense. 

Confidentiality:  ALL  ANSWERS  ARE  STRICTLY  CONFIDENTIAL.  No  one  other  than  Lt  Col 
Alfred  Thai  (research  advisor  at  the  Air  Force  Institute  of  Technology  which  is  an  organization  independent 
of  your  organization),  or  Captain  Joshua  Ligday,  will  ever  see  your  questionnaire.  Findings  will  be 
reported  without  specific  ties  to  individual  names  or  Squadrons.  We  ask  for  some  demographic  and  unit 
information  (rank/grade  and  MAJCOM)  in  order  to  interpret  results  more  accurately,  and  in  order  to  link 
responses  for  an  entire  Command.  Reports  summarizing  trends  in  large  groups  (such  as  MAJCOMs)  may 
be  published. 

Because  this  is  a  web-based  questionnaire,  certain  precautions  have  been  built  into  the  database  to  ensure 
that  your  confidentiality  is  protected.  First,  the  questionnaire  and  database  are  not  stored  on  your 
organization’s  server;  instead,  the  questionnaire  and  database  will  be  stored  on  the  Air  Force  Institute  of 
Technology’s  secure  server.  This  makes  it  impossible  for  your  leaders  to  circumvent  the  surveyors  and 
access  any  identifiable  data  without  their  knowledge.  Second,  only  you  will  have  access  to  your  responses. 
Finally,  the  database  is  protected  by  a  password  that  is  known  only  by  the  aforementioned  surveyors 
making  it  impossible  for  anyone  else  to  access  your  data.  Still,  if  you  don’t  feel  comfortable  completing 
the  on-line  version  of  the  questionnaire  you  may  print  a  paper  version  of  the  questionnaire,  complete  it,  and 
return  it  directly  to  Capt  Joshua  Ligday  at  the  address  listed  below. 

Contact  information:  If  you  have  any  questions  or  comments  about  the  survey,  contact  Captain  Josh 
Ligday  at  the  number,  fax,  mailing  address,  or  e-mail  address  listed  below. 


Captain  Joshua  Ligday 


AFIT/ENV  BLDG  640 

2950  Hobson  Way 

Wright-Patterson  AFB  OH  45433-7765 
Email:  ioshua.ligday@afit.edu 
Phone:  DSN  785-3636  X  6225,  commercial  (937)  427-4362 
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Privacy  Notice 


The  following  information  is  provided  as  required  by  the  Privacy  Act  of  1974: 

Purpose:  The  purpose  of  this  study  is  to  determine  whether  the  proposed  changes  to  the 
current  NAF  ACES-PM  programming  procedures  and  business  rules,  recently  proposed 
by  the  ACES-PM  IPT,  are  efficient  and  complete. 

Routine  Use:  The  survey  results  will  be  used  to  provide  the  ACES-PM  IPT  additional 
insight  into  how  the  NAF  programming  process  can  be  further  improved  in  ACES-PM. 
No  analysis  of  individual  responses  will  be  conducted  and  only  members  of  the  Air  Force 
Institute  of  Technology  research  team  will  be  permitted  access  to  the  raw  data. 

Participation:  Participation  is  VOLUNTARY.  No  adverse  action  will  be  taken  against 
any  member  who  does  not  participate  in  this  survey  or  who  does  not  complete  any  part  of 
the  survey. 


INSTRUCTIONS 

•  Base  your  answers  on  your  own  thoughts  &  experiences 

•  Please  feel  free  to  provide  comments  after  each  question _ 

Demographic  Questions  (Multiple  Choice  &  Short  Answer) 

Dl)  What  is  your  grade/rank? 

D2)  What  is  your  MAJCOM  or  Direct  Reporting  Unit? 

D3)  How  many  years  experience  do  you  have  in  programming? 

D4)  Do  you  have  experience  inputting  NAF  projects  into  ACES-PM? 

A)  If  yes,  approximately  how  many  projects  have  you  inputted? 
(go  onto  question  7) 

B)  If  no,  please  explain  why  you  have  not. 

(you  have  completed  the  survey.) 
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For  questions  G1  through  G23,  please  CIRCLE  A  NUMBER  from  1  to  7  that  BEST 
DESCRIBES  your  opinion  on  the  proposed  ACES-PM  NAF  programming  changes.  If 
no  opinion,  please  circle  N/A.  Each  G  questions  allows  the  participant  to  optional 
information  in  the  comment  section. 


Completely  Strongly  Strongly  Completely 

Disagree  Disagree  Disagree  Undecided  Agree  Agree  Agree 

12  34  5  6  7 


Gl)  I  have  had  adequate  formal  training  in  NAF  project  programming. 

G2)  I  have  had  adequate  formal  training  in  programming  NAF  projects  in  ACES-PM. 

G3)  I  know  the  differences  between  the  O  &  M/MILCON  and  NAF  project 
programming  procedures. 

G4)  I  know  how  to  properly  program  a  NAF  project  into  ACES-PM. 

G5)  If  a  question  arises  concerning  the  input  of  NAF  projects  into  ACES-PM,  I  have 
access  to  the  resources  (user  guides/field  experts)  to  get  the  question  answered. 

The  following  questions  specifically  address  proposed  changes  to  the  current  ACES- 
PM  NAF  programming  procedures  or  issues  that  will  affect  these  changes. 

G6)  It  would  be  beneficial  to  use  automated  emails  via  ACES-PM  to  notify  Base  Level, 
MAJCOM,  and  Air  Staff  project  managers  that  the  status  of  a  particular  project  has 
changed  and/or  an  action  is  required  to  move  the  project  further  in  the  funding  process? 
An  example  would  be  sending  an  email  to  the  MAJCOM  programmer  stating  that  the  Air 
Force  Services  Agency  has  selected  their  project  for  35%  design. 


G7)  It  would  be  beneficial  to  add  a  history  scroll  down  field  and  current  status  field  to  the 
programming  tab  so  all  users  can  easily  see  all  actions  that  have  taken  place  with  the 
project,  who  initiated  them,  and  on  what  date?  An  example  could  be  the  date  the  Design 
Instruction  was  issued,  and  who  approved  the  project  up  to  the  35%  design. 

G8)  When  inputting  a  new  project,  I  always  complete  my  project  manager  information 
under  the  manager's  button  to  include  all  my  contact  information  (phone  #,  email,  etc.). 

G9)  When  the  project  manager  roles  change  for  a  given  project  I  check  to  ensure  that  the 
project  manager  information  is  updated  under  the  "managers"  button. 


97 


G10)  I  would  always  use  the  ACES-PM  milestone  tab  and  update  it  consistently  for  NAF 
projects  if  the  milestones  tab  was  modified  to  better  follow  the  NAF  programming 
process. 

Gil)  When  programming  a  new  NAF  project,  I  make  an  effort  to  determine  if  there  are 
any  companion  projects  (projects  needed  to  complete  the  NAF  project  but  use 
appropriated  funds  (APF)),  and  document  it. 

G12)  It  would  be  beneficial  to  add  a  mandatory  "yes"  and  "no"  check  box  on  the  project 
programming  tab  to  identify  if  a  companion  project(s)  accompanies  the  NAF  project. 

This  would  ensure  a  project  could  not  be  entered  without  identifying  whether  a 
companion  APF  project  exists. 

G13)  If  the  companion  project  "yes'  box  is  checked,  it  would  be  beneficial  to  allow  the 
programmer  to  enter  the  project  number(s)  and  PA(s)  if  known. 

G14)  It  would  be  beneficial  to  add  a  ’pre-INVS’  status  block  to  the  programming  tab  for 
projects  that  currently  do  not  warrant  the  start  of  an  Internal  Needs  Validation  Study 
(INVS). 

G15)  It  would  be  beneficial  to  add  a  data  field  to  the  programming  tab  for  the  INVS  start 
and  completion  date. 


G16)  It  would  be  beneficial  to  allow  the  entire  (INVS)  document  to  be  attached 
electronically  to  the  ACES-PM  project  file  instead  of  only  being  sent  to  the  MAJCOM 
via  a  paper  copy. 

G17)  I  would  not  have  any  complaints  if  Civil  Engineer  Programmers  were  tasked  to 
attach  all  INVS  electronic  documents  into  the  ACES-PM  NAF  project  file. 

G18)  It  would  be  beneficial  to  give  base  level  Services  representatives  read-only  rights 
into  ACES-PM  to  be  able  to  view  all  NAF  projects. 

G19)  For  those  projects  not  selected  for  the  Needs  Assessment  Study  (NAS),  35% 
design,  or  100%  design,  it  would  be  beneficial  if  the  MAJCOM  released  the  rights  back 
to  the  base  and  send  them  an  email  notification. 

G20)  It  would  be  beneficial  to  add  a  check  box  and  to  the  programming  tab  that  shows  if 
a  NAF  project  has  been  approved  at  the  facilities  board  and  a  data  field  for  the  approval 
date. 
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G21)  It  would  be  beneficial  to  revise  the  initial  NAF  programming  fields  with  the 
following  choices: 


Part  A:  Program  Type:  NAF 

Part  B:  Fund  Type:  AAFES 

DECA 

MWR 

Private 

Lodging 

Multi  (will  have  pull  down  menu  to  select  options) 


Part  C:  Type  of  Work:  ADAL 

Construction 
Maintenance/Repair 
Minor  Construction 
O&M 


G22)  It  would  be  beneficial  to  add  the  following  line  items  to  Block  9  of  the  DD  Fonn 
1391: 


Unfunded  Furnishings,  Fixtures,  and  Equipment  (FFE)  (under  the  line  item  “Total 
Request  Rounded” 

Other  Appropriations  (APF)  (  as  the  very  last  line  item) 


G23)  It  would  be  beneficial  to  revise  the  Detailed  Costs  Tab  as  follows: 

Contract  Amount  at  Award 
+  Modifications 
=  Current  Contract  Amount 
+  Contingency  Amount 
=  Subtotal 

+  SIOH  (%  of  subtotal) 

=  Subtotal  (the  PA,  which  will  be  used  in  calculating  the  125%  threshhold) 
+  NAF  Design  Costs 

+  Furnishings,  Fixtures,  and  Equipment  (FFE) 

=  Total  NAF  Investment 
Unfunded  Amount  (APF) 
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Multiple  Choice  Questions 


Ml)  I  currently  fill  out  your  DD  Form  1391  ’s  for  NAF  projects  using: 

A)  The  ACES-PM  template 

B)  A  Microsoft  Word  template 

C)  Both 

D)  Other 

(If  you  do  not  use  ACES  to  generate  the  1391,  please  explain  why) 

M2)  How  often  do  you  check  the  status  and  accuracy  of  your  installation’s  NAF  projects 
in  ACES-PM? 

A)  More  than  once  a  week 

B)  Once  a  week 

C)  More  than  once  a  month 

D)  Once  a  month 

E)  Less  than  once  a  month 

F)  Never 

M3)  How  often  do  other  users  (Services,  wing/group  level  leadership)  ask  you  about  the 
status  of  your  installation’s  NAF  projects? 

A)  More  than  once  a  week 

B)  Once  a  week 

C)  More  than  once  a  month 

D)  Once  a  month 

E)  Less  than  once  a  month 

F)  Never 

M4)  If  the  current  template  is  revised  to  better  fit  the  NAF  process,  will  you  use  the  new 
template  to  input  the  initial  DD  Form  1391  information? 

A)  Yes 

B)  No  (If  no,  please  explain) 

M5  )  Do  you  have  any  other  suggestions  on  how  the  ACES-PM  system  could  be  changed 
in  order  to  better  support  the  NAF  project  programming  process? 

THANK  YOU  FOR  PARTICIPATING 
ALL  INFORMATION  IS  STRICTLY  CONFIDENTIAL 


100 


APPENDIX  C.  EXEMPTION  LETTER 


5  Nov  2003 


MEMORANDUM  FOR  AFIT/ENV 

AFIT/ENR 
AFRL/HEH 
IN  TURN 

FROM:  AFIT/ENV/GEM04 

SUBJECT :  Request  for  Exemption  from  Human  Experimentation  Requirements  (AFI 
40-402):  Thesis  Research,  AFIT/ENV/GEM,  Evaluation  of  Current  Automated  Civil 
Engineering  System  Project  Management  Module  (ACES-PM)  Non- Appropriated  Funds 
(NAF)  Project  Programming  Procedures. 

1 .  Request  exemption  from  Human  Experimentation  Requirements  of  AFI  40-402  for  the 
proposed  questionnaire  of  Base  Level  and  Civil  Engineering  Programmers  to  evaluate 
current  NAF  business  rules  within  ACES  in  conjunction  with  thesis  research  at  the  Air 
Force  Institute  of  Technology.  The  purpose  of  this  study  is  to  determine  whether  the 
proposed  changes  to  the  current  NAF  ACES-PM  programming  procedures  and  business 
rules,  recently  proposed  by  the  ACES-PM  IPT,  are  efficient  and  complete  in  terms  of  the 
whole  Air  Force  wide  programming  perspective.  The  results  of  this  research  effort  will  be 
used  to  give  the  ACES-PM  IPT  recommendations  for  both  additions  and  deletions  to  the 
proposed  changes,  along  with  an  idea  of  how  well  these  changes  are  accepted  within  the 
Air  Force  programming  community. 

2.  This  request  is  based  on  the  Code  of  Federal  Regulations,  title  32,  part  219,  section 
101,  paragraph  (b)  (2);  Research  activities  that  involve  human  subjects  will  be  exempt 
when  the  research  involves  the  use  of  survey  procedures  provided  (i)  information 
obtained  cannot  be  directly  or  through  identifiers  linked  to  the  subjects,  and  (ii) 
disclosure  of  subjects'  responses  does  not  place  the  subjects  at  risk  of  criminal  or  civil 
liability,  financial  strain,  employability  or  reputation  ruin.  The  following  information  is 
provided  to  show  cause  for  such  an  exemption: 

2.1.  Equipment  and  facilities.  No  special  equipment  or  facilities  will  be  used. 

2.2.  Subjects.  Subjects  will  be  officer  or  GS  members  currently  assigned  to 
various  Air  Force  Civil  Engineering  Squadron  programming  elements.  All 
programmers  who  have  experience  inputting  NAF  projects  into  ACES-PM  within 
all  Squadron  programming  elements  will  complete  the  entire  questionnaire 
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2.3.  Timeframe.  Data  will  be  collected  in  between  November  2003  and  March 
2004. 

2.4.  Description  of  the  Questionnaire.  Members  of  programming  elements  will  be 
questioned  at  one  point  in  time.  The  questionnaire  consists  of  33  questions  and 
takes  approximately  15-20  minutes  to  complete.  It  measures  personal  opinions  of 
the  proposed  ACES-PM  business  rule  changes  IAW  NAF  project  programming 
procedures.  The  questionnaire  asks  participants  to  rate  expected  changes  to  the 
project  inputting  process  and  gives  them  a  chance  to  add  comments  for  or  against 
the  changes.  The  questionnaire  will  allow  the  ACES-PM  IPT  to  view  how  well 
their  proposed  changes  are  accepted  in  the  entire  project  programming  community 
and  give  foresight  into  possible  problem  areas  that  they  overlooked.  Each 
question  will  have  a  comment  section  to  allow  the  participant  to  justify  each 
answer  and  to  ensure  that  new  ideas  are  captured  with  the  questionnaire. 

Data  are  collected  via  a  web-based  survey  that  will  be  sent  out  to  each  MAJCOM 
Civil  Engineering  program  office,  and  distributed  to  each  base  programming 
office.  Programmers  will  be  given  advance  notice  of  the  questionnaire  and  data 
collection  from  their  supervisors. 

To  ensure  the  anonymity  of  the  participants,  certain  precautions  are  built  into  the 
database  used  to  collect  the  data  with  the  web-based  questionnaire.  First,  the 
questionnaire  and  database  are  not  stored  on  any  of  the  participating 
organizations’  servers;  instead,  the  questionnaire  and  database  are  stored  on  the 
Air  Force  Institute  of  Technology’s  secure  server.  This  makes  it  impossible  for 
leaders  from  participating  organizations  to  circumvent  the  researcher  and  try  to 
access  any  identifiable  data  without  the  researcher’s  knowledge.  Second, 
participants’  access  to  the  questionnaire  is  limited  to  only  their  responses. 

Finally,  the  database  is  protected  by  a  password  that  is  known  only  by  the 
researcher  making  it  impossible  to  access  data.  Still,  organizational  members  that 
do  not  feel  comfortable  completing  an  on-line  version  of  the  questionnaire  will 
offered  the  option  to  print  a  traditional  paper  version  of  the  questionnaire  so  that 
they  can  complete  it  and  return  it  directly  to  the  researcher  by  mail. 

In  addition,  all  participants  will  be  thoroughly  briefed  on  the  project’s  objective 
and  their  role  prior  to  any  participation.  In  addition,  there  is  no  deception 
involved  in  this  study.  Participants  are  told  that  the  researcher  is  interested  in 
exactly  what  is  being  asked  and  only  that.  Thus,  the  researcher  does  not  try  to 
“read  between  the  lines”  of  any  infonnation  provided  by  participants. 

2.5.  Data  collected.  No  identifying  information  is  obtained  through  the  survey. 

2.6.  Informed  consent:  All  subjects  are  self-selected  to  volunteer  to  participate  in 
the  questionnaire.  No  adverse  action  is  taken  against  those  who  choose  not  to 
participate.  Subjects  are  made  aware  of  the  nature  and  purpose  of  the  research, 
sponsors  of  the  research,  and  disposition  of  the  survey  results.  A  copy  of  the 
Privacy  Act  Statement  of  1974  is  presented  for  their  review. 
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Because  the  message  inviting  participation  comes  from  the  member’s  supervisor, 
there  may  be  some  risk  of  coercion.  However,  the  letter  inviting  participation 
stresses  the  decision  to  participate  is  voluntary.  In  addition,  the  questionnaire’s 
instruction  states,  “Your  participation  is  COMPLETELY  VOLUNTARY. 
However,  your  input  is  important  for  us  to  understand  the  pattern  of  voluntary 
turnover.  You  may  withdraw  from  this  study  at  any  time.  Your  decision  to 
participate  or  withdraw  will  not  jeopardize  your  relationship  with  your 
organization,  the  Air  Force  Institute  of  Technology,  the  Air  Force,  or  the 
Department  of  Defense.” 

2.7.  Risks  to  Subjects:  Individual  responses  of  the  subjects  will  not  be  disclosed. 
Answers  will  only  be  grouped  according  to  MAJCOMs,  which  will  keep  the 
anonymity  of  the  participants  intact.  This  eliminates  any  risks  to  the  subjects  as 
noted  in  paragraph  2.  There  are  no  anticipated  medical  risks  associated  with  this 
study. 

3.  If  you  have  any  questions  about  this  request,  please  contact  Captain  Josh  Ligday  - 
Phone  255-3636,  ext.  4225;  E-mail  -  Joshua.Ligday@afit.edu  or  Lieutenant  Colonel 
Alfred  E.  Thai  Jr.  who  will  serve  as  the  Faculty  Advisor  (primary  investigator)  -  Phone 
255-3636,  ext.  4798;  E-mail  -  Alfred.Thal@afit.edu. 


//signed// 

JOSHUA  C.  LIGDAY,  Capt,  USAF 
Graduate  Student,  AFIT/ENV/GEM04 


//signed// 

ALRED  E.THAL,  JR.,  Lt  Col,  USAF 
Assistant  Professor  of  Management 
Faculty  Advisor,  AFIT/ENV/GEM 
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APPENDIX  D.  SURVEY  DISSEMINATION  EMAIL 


FROM:  LIGDAY  JOSHUA  C  CAPT/ENV 

TO:  MAJCOM  CEP  CORP  BOXES 

SUBJECT :  Request  for  Dissemination  of  AFIT  Thesis  Survey 


MAJCOMs: 

I  am  a  graduate  student  at  AFIT  doing  my  thesis  on  improving  the  NAF  programming  process  in 
ACES-PM.  I  am  surveying  all  Base  Level  programmers  to  determine  if  the  proposed  NAF 
changes  to  ACES-PM,  recently  proposed  by  a  sub  ACES-PM  IPT  NAF  users  meeting  in  Oct  03, 
are  efficient  and  complete.  The  sub  IPT  met  to  come  up  with  new  business  rules  that  improve 
NAF  programming  in  ACES-PM.  The  results  of  this  research  effort  will  be  used  to  give  to  the 
ACES-PM  IPT  recommendations  for  both  additions  and  deletions  to  the  proposed  changes,  along 
with  an  idea  of  how  well  these  changes  will  be  accepted  within  the  Air  Force  programming 
community.  The  33  question  survey  addresses  the  following  issues:  NAF  programming 
knowledge  and  training,  knowledge  and  training  of  NAF  programming  in  ACES-PM,  specific 
proposed  changes  to  the  NAF  programming  process  in  ACES-PM,  and  current  programming 
practices  that  might  affect  how  well  the  proposed  changes  work.  The  feedback  that  is  provided 
in  this  questionnaire  will  directly  aid  in  the  improvement  of  NAF  project  programming 
throughout  the  USAF. 

The  sponsor  for  this  research  is  Mr.  William  Marsh,  HQ  AFMC/CEPD,  the  chairman  of  the  ACES- 
PM  IPT. 

I  need  your  help  in  forwarding  this  survey  (see  survey  link  below)  to  your  installation’s 
programming  offices.  The  survey  only  takes  about  15-20  minutes  at  most  to  complete, 
guarantees  anonymity,  and  is  completely  voluntary  for  the  participants.  Feel  free  to  pass  the 
survey  on  to  those  you  know  who  have  had  recent  base  level  ACES-PM  NAF  programming 
experience. 

Please  let  me  or  my  thesis  advisor  know  if  you  have  any  questions,  comments,  or  concerns  about 
this  survey  and  its  intended  purpose. 

Also,  please  send  me  an  email  once  you  have  forwarded  the  link  to  the  required  offices. 

Thanks  for  your  time  and  help  in  this  research  effort.  The  results  will  directly  help  the  ACES  PM 
IPT  make  the  NAF  programming  experience  better  for  everyone! 


http://en.afit.edu/Survevs/Liqday/ 


Capt  Joshua  C.  Ligday 
Graduate  Student,  AFIT/ENV/GEM04 
ioshua.liqdav@afit.edu  (more  preferred) 
Phone  255-3636,  ext.  4225 

Lt  Col  Alfred  E  Thai,  Jr 
Thesis  Advisor,  AFIT/ENV/GEM 
alfred.thal@afit.edu 
Phone  255-3636,  ext.  4798 
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VITA 


Josh  Ligday  was  born  in  St.  Paul,  Minnesota  and  attended  Stillwater  Senior  High 
School.  He  studied  at  the  University  of  Saint  Thomas  Minnesota  for  his  first  three  years 
of  college,  and  transferred  to  the  University  of  Minnesota-Twin  Cities  for  his  last  two 
years.  Upon  completion  of  his  studies  he  earned  a  Bachelor  of  Arts  in  General  Studies 
from  the  University  of  Saint  Thomas,  and  a  Bachelor  of  Science  in  Civil  Engineering 
from  the  University  of  Minnesota.  After  commissioning,  Lieutenant  Ligday  was 
assigned  to  the  5th  Civil  Engineer  Squadron  at  Minot  AFB,  North  Dakota.  While  at 
Minot  he  worked  as  a  Pavements  Engineer,  Programmer,  and  Chief  of  Construction 
Management.  Lieutenant  Ligday  was  then  assigned  to  the  Air  Force  Institute  of 
Technology  to  earn  of  Masters  of  Science  in  Engineering  Management.  He  was  also 
promoted  to  Captain  while  attending  graduate  school.  His  next  assignment  after 
graduation  will  be  the  Chief  of  Maintenance  Engineering,  60th  Civil  Engineer  Squadron, 
Travis  AFB,  California. 
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Form  Approved 

REPORT  DOCUMENTATION  PAGE  omb  no.  074-0188 

The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering 
and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  the  collection  of 
information,  including  suggestions  for  reducing  this  burden  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188), 

1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  an  penalty 
for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  OMB  control  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. _ _ 

1.  REPORT  DATE  (DD-MM-YYYY)  2.  REPORT  TYPE  3.  DATES  COVERED  (From  -  To) 

23-03-2004  Master’s  Thesis  SEP  2002  -  MAR  2004 


4.  TITLE  AND  SUBTITLE 

5a.  CONTRACT  NUMBER 

EVALUATION  OF  CURRENT  AUTOMATED  CIVIL  ENGINEER  SYSTEM  NON-APPROPRIATED 
FUNDS  PROJECT  PROGRAMMING  PROCEDURES 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

If  funded,  enter  ENR  # 

Ligday,  Joshua,  C.,  Captain,  USAF 
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5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAMES(S)  AND  ADDRESS(S)  8.  PERFORMING  ORGANIZATION 

Air  Force  Institute  of  Technology  REPORT  NUMBER 

Graduate  School  of  Engineering  and  Management  (AFIT/EN) 

2950  Hobson  Way,  Building  641  AFIT/GEM/ENV/04M- 1 2 

WPAFB  OH  45433-7765 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES)  10.  SPONSOR/MONITOR’S 

HQ  AFMC/CEPD  ACRONYM(S) 

William  T.  Marsh 

4225  Logistics  Avenue  11.  SPONSOR/MONITOR’S  REPORT 

WPAFB,  OH  45433-5746  NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

It  is  essential  that  businesses  continually  improve  their  automated  information  systems  (AIS)  to  support  the  changing  needs  of  the 
organization.  The  Air  Force  civil  engineering  organization  is  no  exception  since  they  have  drastically  improved  their  Automated  Civil 
Engineer  System  (ACES)  through  end  user  support  since  its  implementation  in  2000.  However,  there  are  many  problems  associated  with  the 
non-appropriated  funds  (NAF)  project  programming  business  rules  within  ACES.  These  problem  areas  were  not  addressed  until  recently 
when  an  integrated  process  team  met  and  proposed  numerous  changes  to  how  NAF  programming  is  accomplished  in  ACES.  This  research 
effort,  through  a  web-based  survey,  focuses  on  the  perceived  benefits  of  these  proposed  changes  from  a  base-level  programming  perspective. 
It  also  investigated  current  programming  procedures  that  might  affect  how  well  the  proposed  changes  are  implemented  along  with  NAF  and 
ACES  training  issues.  Descriptive  statistics  were  used  to  answer  the  research  questions  using  survey  responses  from  a  sample  size  of  35 
base-level  programmers. 

The  results  indicated  that  programmers  “agree”  or  “strongly  agree”  that  the  majority  of  proposed  changes  will  be  beneficial  in 
improving  NAF  programming  in  ACES.  However,  several  potential  problems  areas  might  surface,  due  to  current  programming  procedures 
at  base-level,  when  these  changes  are  implemented  into  ACES.  Automatic  email  notifications  on  project  status,  electronic  attachments  to  the 
project  file,  and  use  of  non  ACES-PM  templates  are  all  areas  of  concern  brought  up  in  this  research  effort. 
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